EEL 5544 - Noise in LinearSystems

(Should be called ldquoProbability andRandom Processes for Electrical

Engineersrdquo)

Dr John M Shea

Fall 2008

EEL5544

Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition

Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition

Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required

EEL5544 S-1

Meeting Time 935ndash1025 MondayWednesdayFriday

Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition

Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition

Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required

EEL5544 S-1

Meeting Time 935ndash1025 MondayWednesdayFriday

Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition

Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required

EEL5544 S-1

Meeting Time 935ndash1025 MondayWednesdayFriday

Meeting Time 935ndash1025 MondayWednesdayFriday

Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201

Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon

Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon

E-mail jsheaeceufledu

Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon

E-mail jsheaeceufleduInstant messaging On AIM eel5544

Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon

E-mail jsheaeceufleduInstant messaging On AIM eel5544

Class Web page httpwirelesseceufledueel5544

Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon

E-mail jsheaeceufleduInstant messaging On AIM eel5544

Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea

EEL5544 S-2

Office 439 New Engineering Building

Office 439 New Engineering BuildingPhone (352)846-3042

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)

EEL5544 S-3

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Office 439 New Engineering BuildingPhone (352)846-3042

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)

EEL5544 S-3

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)

EEL5544 S-3

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays

Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)

EEL5544 S-3

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processes

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Suggested References

bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)

Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)

bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)

EEL5544 S-4

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grader Byonghyok Choi (aru0080ufledu)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grader Byonghyok Choi (aru0080ufledu)

Course Topics (as time allows)

Goals and Objectives

EEL5544 S-5

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc

EEL5544 S-6

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Because of limited grading support most homeworksets will not be graded for on-campus students

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade

EEL5544 S-7

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems

When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem

EEL5544 S-8

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Attendance Attendance is not mandatory

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly

EEL5544 S-9

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Academic Honesty

All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action

This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others

EEL5544 S-10

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Honor statements on tests must be signed in orderto receive any credit for that test

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Honor statements on tests must be signed in orderto receive any credit for that test

Collaboration on homework is permitted unlessexplicitly prohibited provided that

1 Collaboration is restricted to students currently inthis course

2 Collaboration must be a shared effort

3 Each student must write up hisher homeworkindependently

EEL5544 S-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

4 On problems involving MATLAB programs eachstudent should write their own program

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs

EEL5544 S-12

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

I have a zero-tolerance policy for cheating in thisclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code

EEL5544 S-13

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Noise in Linear Systems Lecture 1

RANDOM EXPERIMENTS

What is a random experiment

QWhat do we mean by random

Output is unpredictable in some sense

EEL5544 L1-1

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN A random experiment

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions

A random experiment is specified by

1 an experimental procedure2 a set of outcomes and measurement restrictions

EEL5544 L1-2

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An outcome

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results

DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω

EEL5544 L1-3

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Tossing a coin

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded

bull Q What are the outcomes

Ω =heads tails

EEL5544 L1-4

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided die

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed

bull QWhat are the outcomes

Ω =123456

EXA 6-sided die is rolled and whether the topface is even or odd is observed

bull QWhat are the outcomes

Ω =even odd

EEL5544 L1-5

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

If the outcome of an experiment is random how canwe apply quantitative techniques to it

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

If the outcome of an experiment is random how canwe apply quantitative techniques to it

This is the theory of probability

EEL5544 L1-6

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12

1 Probability as Intuition

2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)

3 Probability as a Measure of Frequency of Occurrence

4 Probability Based on Axiomatic Theory

EEL5544 L1-7

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)

bull Some experiments are said to be ldquofairrdquo

ndash For a fair coin or a fair die toss the outcomes areequally likely

ndash Equally likely outcomes are common in manysituations

bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis

bull Given an experiment with equally likely outcomeswe can determine the probabilities easily

EEL5544 L1-8

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes

ndash An outcome or set of outcomes with probability 1(one) is certain to occur

ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur

ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points

EEL5544 L1-9

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes

EX Tossing a fair coin

Let pH = Prob heads pT = Prob tails

Then pH + pT = 1(the probability that something occurs is 1)

Since pH = pT pH = pT = 12

EEL5544 L1-10

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX Rolling a fair 6-sided die

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX Rolling a fair 6-sided die

Let pi = Prob top face is i

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX Rolling a fair 6-sided die

Let pi = Prob top face is iThen

6sumi=1

pi = 1

rArr 6p1 = 1

rArr p1 =16

So pi = 16 for i = 1 2 3 4 5 6

EEL5544 L1-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes

EX Rolling a fair 6-sided die

What is Prob 1 or 2

Prob 1 or 2 = Prob 1+ Prob 2

=16

+16

=13

EEL5544 L1-12

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The basic principle of counting

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes

EEL5544 L1-13

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We can use equally-likely outcomes on repeatedtrials but we have to be careful

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We can use equally-likely outcomes on repeatedtrials but we have to be careful

EX Rolling a fair 6-sided dice twice

Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die

EEL5544 L1-14

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL5544 L1-15

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

What are some problems with defining probability inthis way

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

What are some problems with defining probability inthis way

1 Requires equally likely outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomes

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

What are some problems with defining probability inthis way

1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena

2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness

EEL5544 L1-16

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as a Measure of Frequency ofOccurrence

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Probability as a Measure of Frequency ofOccurrence

bull Consider a random experiment that has K possibleoutcomes K lt infin

bull Let Nk(n) = the number of times the outcome is k

bull Then we can tabulate Nk(n) for various values of k

and n

bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo

EEL5544 L1-17

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN The relative frequency

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN The relative frequency of outcome k of arandom experiment is

fk(n) =Nk(n)

n

bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value

EEL5544 L1-18

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An experiment possesses statisticalregularity

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFN An experiment possesses statisticalregularity if

limnrarrinfin

fk(n) = pk (a constant)forallk

DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k

EEL5544 L1-19

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

PROPERTIES OF RELATIVE FREQUENCY

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

PROPERTIES OF RELATIVE FREQUENCY

bull Note that

0 le Nk(n) le nforallk

because Nk(n) is just the of times outcome k

occurs in n trials

Dividing by n yields

0 le Nk(n)n

= fk(n) le 1 forallk = 1 K (1)

EEL5544 L1-20

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull If 1 2 K are all of the possible outcomes

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull If 1 2 K are all of the possible outcomes then

Ksumk=1

Nk(n) = n

Again dividing by n yields

Ksumk=1

fk(n) = 1 (2)

EEL5544 L1-21

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Consider the event E that an even number occurs

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Consider the event E that an even number occurs

What can we say about the number of times E isobserved in n trials

NE(n) = N2(n) + N4(n) + N6(n)

What have we assumed in developing this equation

EEL5544 L1-22

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General property

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

Then dividing by n

fE(n) =NE(n)

n=

N2(n) + N4(n) + N6(n)n

= f2(n) + f4(n) + f6(n)

bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then

fC(n) = fA(n) + fB(n) (3)

EEL5544 L1-23

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull In some sense we can define the probability of anevent as its long-term relative frequency

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull In some sense we can define the probability of anevent as its long-term relative frequency

What are some problems with this definition

1 It is not clear when and in what sense the limitexists

2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly

3 We cannot use this definition if the experimentcannot be repeated

EEL5544 L1-24

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We need a mathematical model ofprobability that is not based on a particularapplication or interpretation

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should

1 be useful for solving real problems

2 agree with our interpretation of probabilityas relative frequency

3 agree with our intuition (whereappropriate)

EEL5544 L2-1

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elements

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the set

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EEL 5544 Lecture 2PROBABILITY SPACES

We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )

bull The elements of a probability space for a randomexperiment are

1 DEFN The sample space denoted by Ω is the setof all outcomes

EEL5544 L2-2

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The sample space is also known as the universal setor reference set

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

The sample space is also known as the universal setor reference set

Sample spaces come in two basic varieties discreteor continuous

DEFN A discrete set is either finite or countablyinfinite

DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers

EEL5544 L2-3

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Examplesbull The integers Z

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Examplesbull The integers Zbull Positive integers Z+

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Examplesbull The integers Zbull Positive integers Z+

bull The rationals Q

DEFN A continuous set is not countable

DEFN If a and b gt a are in an interval I then ifa le x le b x isin I

EEL5544 L2-4

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Intervals can be either open closed or half-open

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a

and bndash An open interval (a b) does not contain the

endpoints a and bndash An interval can be half-open such as (a b] which

does not contain a or [a b) which does notcontain b

bull Intervals can also be either finite infinite orpartially infinite

EEL5544 L2-5

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Example of continuous sets

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Example of continuous setsbull the real line R

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbers

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

EX

Example of continuous setsbull the real line Rbull the transcendental numbers (ie the

numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite

EEL5544 L2-6

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is even

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like

ndash What is the probability that the result is evenndash What is the probability that the result is le 2

EEL5544 L2-7

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFNEvents are sets of outcomes to which weassign probability

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top face

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

DEFNEvents are sets of outcomes to which weassign probability

Examples based on previous examples ofsample spaces

(a) EX

Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes

L = 1 2 3 4

EEL5544 L2-8

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(b) EX

Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events

E = 2 4 6 and O = 1 3 5

EEL5544 L2-9

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(c) EX

Toss a coin 3 times and note the sequenceof outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(c) EX

Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes

A1 = HHHHTH HHT HTT

EEL5544 L2-10

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(d) EX

Toss a coin 3 times and note the number ofheads

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11

(d) EX

Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes

O = 1 3

EEL5544 L2-11