ECE302 – Probability and Applications - Tutorial #1rrafie/ece302/Tutorial_01.pdf · 2017. 9....

ECE302 – Probability and ApplicationsTutorial #1

Reza Rafie

rrafie@ece.utoronto.ca

Administrivia

Reza RafiePhD Student in Communications

1

Administrivia

• Starting from today, you have an assignment at theend of the tutorial. You have assignments everyother week.

• Starting from next week, you have a quiz at the endof the tutorial. You have quizzes every other week.

• If you have not yet, check the course information onblackboard for information about quizzes andassignments.

2

Administrivia

• Starting from today, you have an assignment at theend of the tutorial. You have assignments everyother week.

• Starting from next week, you have a quiz at the endof the tutorial. You have quizzes every other week.

• If you have not yet, check the course information onblackboard for information about quizzes andassignments.

2

Administrivia

• Starting from today, you have an assignment at theend of the tutorial. You have assignments everyother week.

• Starting from next week, you have a quiz at the endof the tutorial. You have quizzes every other week.

• If you have not yet, check the course information onblackboard for information about quizzes andassignments.

2

Administrivia

• An unexplained missed quiz or assignment willcount as a zero towards your final mark. You musttake all assignments and quizzes.

• If you are unable to attend a tutorial, ask for yourprofessor’s permission.

3

Administrivia

• An unexplained missed quiz or assignment willcount as a zero towards your final mark. You musttake all assignments and quizzes.

• If you are unable to attend a tutorial, ask for yourprofessor’s permission.

3

Illustrating randomness and the role of probabilitytheory in our lives

A student lives in Mississauga and during the term leavesan hour before her first class of the day. However, duringthe exam period she leaves an hour and 15 mins beforeher first exam of the day. Why (think probability of beinglate)?

The probability of being late is a “decreasing”function of the amount of time she allots for commuting.

4

Illustrating randomness and the role of probabilitytheory in our lives

The probability of being late is a “decreasing” function ofthe amount of time she allots for commuting.

4

Illustrating randomness and the role of probabilitytheory in our lives

Climate change has meant that the planet is gettingwarmer, but we’ve had a crazy cold summer. Does thismean climate change is a hoax?

Uh, no! While the meanor average temperature has gone up, there’s alsoincreasing variation in the weather!

5

Illustrating randomness and the role of probabilitytheory in our lives

Uh, no! While the mean or average temperature has goneup, there’s also increasing variation in the weather!

5

Illustrating randomness and the role of probabilitytheory in our lives

Sports analytics is a growing business: before signing aplayer (with millions at stake) teams must predict how aparticular player will perform. How do they do that? Isthe player’s past performance enough?

Clearly not (else,we will still be trying to sign Michael Jordan, not StephCurry).

6

Illustrating randomness and the role of probabilitytheory in our lives

Clearly not (else, we will still be trying to sign MichaelJordan, not Steph Curry).

6

Illustrating randomness and the role of probabilitytheory in our lives

Communications depends heavily on probability theory:think of a cellphone voice call. What is being said israndom. Where the call is made from is random (thesystem does not control where you are when you placethe call). Importantly, the signal must travel from the cellphone to a basestation—loss in power due to buildings,trees and other obstacles between the cellphone andbasestation is random (in fact, it is largely because ofthis random loss in power that data rates for cellphonesare much lower than for a wired connection).

7

Question 1.1

Consider the following three random experiments:Experiment 1: Toss a coin.Experiment 2: Toss a die.Experiment 3: Select a ball at random from an urncontaining balls numbered 0 to 9.

8

(a) Specify the sample space of each experiment.

9

Question 1.1

Each random experience has an unpredictable outcome.

Possible outcomes of rolling a coin are:

B and Z

10

Question 1.1

Each random experience has an unpredictable outcome.Possible outcomes of rolling a coin are:

B and Z

10

Question 1.1

Each random experience has an unpredictable outcome.Possible outcomes of rolling a coin are:

B

and Z

10

Question 1.1

Each random experience has an unpredictable outcome.Possible outcomes of rolling a coin are:

B and Z

10

Question 1.1

The set of all possible outcomes is called the samplespace.

The sample space of rolling a coin is:

S1 =

B , Z

11

Question 1.1

The set of all possible outcomes is called the samplespace. The sample space of rolling a coin is:

S1 =

B , Z

11

Question 1.1

The sample space of rolling a dice is:

S2 ={

, , , , ,

}

12

Question 1.1

The sample space for experiment 3 is:

S3 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

13

Question 1.1

(b) Find the relative frequency of each outcome in eachof the above experiments in a large number ofrepetitions of the experiment. Explain your answer.

14

Question 1.1

Suppose that a discrete experiment is repeated n timesunder identical conditions. If we show the number ofoccurrences of the ith outcome by Ni(n), the relativefrequency of the outcome is defined by

fi(n) =Ni(n)n

15

Question 1.1

For rolling a coin: fh(n) ≈ 0.5 and ft(n) ≈ 0.5.

For rolling adice: fi(n) ≈ 1

6 , for i ∈ {1, 2, . . . , 6}. For Experiment 3:fi(n) ≈ 1

10 , for i ∈ {0, 1, 2, . . . , 9}. Note that for all of theseexperiments we assumed a uniform distribution over allpossible outcomes, that is, the coin is unbiased, the diceis fair, and all of the balls in the urn are identical.

16

Question 1.1

For rolling a coin: fh(n) ≈ 0.5 and ft(n) ≈ 0.5. For rolling adice: fi(n) ≈ 1

6 , for i ∈ {1, 2, . . . , 6}.

For Experiment 3:fi(n) ≈ 1

10 , for i ∈ {0, 1, 2, . . . , 9}. Note that for all of theseexperiments we assumed a uniform distribution over allpossible outcomes, that is, the coin is unbiased, the diceis fair, and all of the balls in the urn are identical.

16

Question 1.1

For rolling a coin: fh(n) ≈ 0.5 and ft(n) ≈ 0.5. For rolling adice: fi(n) ≈ 1

6 , for i ∈ {1, 2, . . . , 6}. For Experiment 3:fi(n) ≈ 1

10 , for i ∈ {0, 1, 2, . . . , 9}.

Note that for all of theseexperiments we assumed a uniform distribution over allpossible outcomes, that is, the coin is unbiased, the diceis fair, and all of the balls in the urn are identical.

16

Question 1.1

For rolling a coin: fh(n) ≈ 0.5 and ft(n) ≈ 0.5. For rolling adice: fi(n) ≈ 1

6 , for i ∈ {1, 2, . . . , 6}. For Experiment 3:fi(n) ≈ 1

10 , for i ∈ {0, 1, 2, . . . , 9}. Note that for all of theseexperiments we assumed a uniform distribution over allpossible outcomes, that is, the coin is unbiased, the diceis fair, and all of the balls in the urn are identical.

16

Question 2.2

A dice is tossed twice and the number of dots facing upin each toss is counted and noted in the order ofoccurrence.(a) Find the sample space.

17

Question 2.2

What are the possible outcomes?

...

18

Question 2.2

What are the possible outcomes?

...

18

Question 2.2

What are the possible outcomes?

...

18

Question 2.2

What are the possible outcomes?

...

18

Question 2.2

What are the possible outcomes?

...

18

Question 2.2

The sample space is

S = {11, 12, 13, . . . , 16, 21, 22, . . . , 66}

1st

2nd

19

Question 2.2

The sample space is

S = {11, 12, 13, . . . , 16, 21, 22, . . . , 66}

|S| =?

1st

2nd

19

Question 2.2

The sample space is

S = {11, 12, 13, . . . , 16, 21, 22, . . . , 66}

|S| = 36

1st

2nd

19

Question 2.2

The sample space is

S = {11, 12, 13, . . . , 16, 21, 22, . . . , 66}

1st

2nd

19

Question 2.2

(b) Find the set A corresponding to the event “number ofdots in first toss is not less than number of dots insecond toss.”

20

Question 2.2

Events are some subsets of the sample space that wewould like to assign them a probability.

21

Question 2.2

• Every event is a subset of the sample space.

• Not all of the subsets are necessarily events (we talkmore about it later in the course).

Fact:In this course, if the sample space is discrete, then allof the subsets of the sample space are usually events!

22

Question 2.2

• Every event is a subset of the sample space.• Not all of the subsets are necessarily events (we talkmore about it later in the course).

Fact:In this course, if the sample space is discrete, then allof the subsets of the sample space are usually events!

22

Question 2.2

• Every event is a subset of the sample space.• Not all of the subsets are necessarily events (we talkmore about it later in the course).

Fact:In this course, if the sample space is discrete, then allof the subsets of the sample space are usually events!

22

Question 2.2

1st

2nd

23

Question 2.2

1st

2nd

23

Question 2.2

1st

2nd

23

Question 2.2

A = {11, 21, 22, 31, 32, 33, 41, . . . , 65, 66}

|A| = 21

24

Question 2.2

A = {11, 21, 22, 31, 32, 33, 41, . . . , 65, 66}

|A| =

21

24

Question 2.2

A = {11, 21, 22, 31, 32, 33, 41, . . . , 65, 66}

|A| = 21

24

Question 2.2

(c) Find the set B corresponding to the event “number ofdots in first toss is 6.”

25

Question 2.2

1st

2nd

26

Question 2.2

1st

2nd

26

Question 2.2

1st

2nd

26

Question 2.2

B = {61, 62, 63, 64, 65, 66}

|B| = 6

27

Question 2.2

B = {61, 62, 63, 64, 65, 66}

|B| =

6

27

Question 2.2

B = {61, 62, 63, 64, 65, 66}

|B| = 6

27

Question 2.2

(d) Does A imply B or does B imply A?

28

Question 2.2

If the event B happens it means that the outcome x is inB, that is x ∈ B.

Note that, B ⊂ A =⇒ x ∈ A.So, if B occurs, A occurs too!Therefore B implies A.

29

Question 2.2

If the event B happens it means that the outcome x is inB, that is x ∈ B.Note that, B ⊂ A =⇒ x ∈ A.

So, if B occurs, A occurs too!Therefore B implies A.

29

Question 2.2

If the event B happens it means that the outcome x is inB, that is x ∈ B.Note that, B ⊂ A =⇒ x ∈ A.So, if B occurs, A occurs too!

Therefore B implies A.

29

Question 2.2

If the event B happens it means that the outcome x is inB, that is x ∈ B.Note that, B ⊂ A =⇒ x ∈ A.So, if B occurs, A occurs too!Therefore B implies A.

29

Question 2.2

(e) Find A ∩ Bc and describe this event in words.

30

Question 2.2

Bc is the complement of B.

So, A∩ Bc is the event that A occurs and B does not occur.

In wordsNumber of dots in first toss is not less than number ofdots in second toss and number of dots in first toss isnot 6.

31

Question 2.2

Bc is the complement of B.So, A∩ Bc is the event that A occurs and B does not occur.

In wordsNumber of dots in first toss is not less than number ofdots in second toss and number of dots in first toss isnot 6.

31

Question 2.2

Bc is the complement of B.So, A∩ Bc is the event that A occurs and B does not occur.

In wordsNumber of dots in first toss is not less than number ofdots in second toss and number of dots in first toss isnot 6.

31

Question 2.2

1st

2nd

32

Question 2.2

A ∩ Bc = {11, 21, 22, 31, 32, 33, 41, 42, 43, 44, 51, 52, 53, 54, 55}

33

Question 2.2

(f ) Let C corresponds to the event “number of dots indices differs by 2.” Find A ∩ C.

34

Question 2.2

A ∩ C =Number of dots in first toss is not less thannumber of dots in second toss

andnumber of dots in dices differs by 2.

A ∩ C = {31, 42, 53, 64}

35

Question 2.2

A ∩ C =Number of dots in first toss is not less thannumber of dots in second toss

andnumber of dots in dices differs by 2.

A ∩ C = {31, 42, 53, 64}

35

Question 2.4

A binary communication system transmits a signal X thatis either a +2 voltage signal or a −2 voltage signal. Amalicious channel reduces the magnitude of thereceived signal by the number of heads it counts in twotosses of a coin. Let Y be the resulting signal.(a) Find the sample space.

36

Question 2.4

Number of heads (N) in two tosses of a coin can be 0,1 or2.

We know that |Y| = |X| − N in which N ∈ {0, 1, 2}.If X = +2, possible outcomes are Y = +2, +1 or 0.If X = −2, possible outcomes are Y = -2, -1 or 0.Therefore, SY = {−2,−1, 0,+1,+2}.

37

Question 2.4

Number of heads (N) in two tosses of a coin can be 0,1 or2.We know that |Y| = |X| − N in which N ∈ {0, 1, 2}.

If X = +2, possible outcomes are Y = +2, +1 or 0.If X = −2, possible outcomes are Y = -2, -1 or 0.Therefore, SY = {−2,−1, 0,+1,+2}.

37

Question 2.4

Number of heads (N) in two tosses of a coin can be 0,1 or2.We know that |Y| = |X| − N in which N ∈ {0, 1, 2}.If X = +2, possible outcomes are Y = +2, +1 or 0.

If X = −2, possible outcomes are Y = -2, -1 or 0.Therefore, SY = {−2,−1, 0,+1,+2}.

37

Question 2.4

Number of heads (N) in two tosses of a coin can be 0,1 or2.We know that |Y| = |X| − N in which N ∈ {0, 1, 2}.If X = +2, possible outcomes are Y = +2, +1 or 0.If X = −2, possible outcomes are Y = -2, -1 or 0.

Therefore, SY = {−2,−1, 0,+1,+2}.

37

Question 2.4

Number of heads (N) in two tosses of a coin can be 0,1 or2.We know that |Y| = |X| − N in which N ∈ {0, 1, 2}.If X = +2, possible outcomes are Y = +2, +1 or 0.If X = −2, possible outcomes are Y = -2, -1 or 0.Therefore, SY = {−2,−1, 0,+1,+2}.

37

Question 2.4

(b) Find the set of outcomes corresponding to the event“transmitted signal was definitely +2”

38

Question 2.4

We already saw that if X = +2, possible outcomes are Y =

+2, +1 or 0.

Also, we saw that Y = 0 can happen when X = −2.Thus, as a set, the answer is {+1,+2}.

39

Question 2.4

We already saw that if X = +2, possible outcomes are Y =

+2, +1 or 0.Also, we saw that Y = 0 can happen when X = −2.

Thus, as a set, the answer is {+1,+2}.

39

Question 2.4

We already saw that if X = +2, possible outcomes are Y =

+2, +1 or 0.Also, we saw that Y = 0 can happen when X = −2.Thus, as a set, the answer is {+1,+2}.

39

Question 2.4

X = +2

X = −2

Y

+2

+1

0

−1

−2

40

Question 2.4

(c) Describe in words the event corresponding to theoutcome Y = 0

41

Question 2.4

This is the case that N = 2.

We cannot determine the input!

42

Question 2.4

This is the case that N = 2.We cannot determine the input!

42

Question 2.5

A desk drawer contains six pens, four of which are dry.(a) The pens are selected at random one by one until agood pen is found. The sequence of test results is noted.What is the sample space?

43

Question 2.5

The first pen is good!

Sa = {G

,DG,DDG,DDDG,DDDDG}

44

Question 2.5

The first pen is dry, the second pen is good!

Sa = {G,DG,

DDG,DDDG,DDDDG}

44

Question 2.5

We can select at most 4 dry pens!

Sa = {G,DG,DDG,DDDG,DDDDG}

44

Question 2.5

(b) Suppose that only the number, and not the sequence,of pens tested in part (a) is noted. Specify the samplespace.

45

Question 2.5

Sa = {G,DG,DDG,DDDG,DDDDG}

Sb = {1, 2, 3, 4, 5}

46

Question 2.5

Sa = {G,DG,DDG,DDDG,DDDDG}

Sb = {1, 2, 3, 4, 5}

46

Question 2.5

(c) Suppose that the pens are selected one by one andtested until both good pens have been identified, andthe sequence of test results is noted. What is the samplespace?

47

Question 2.5

Sc = {GG

,GDG,DGG,GDDG, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG

,DGG,GDDG, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG,DGG

,GDDG, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG,DGG,GDDG

, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

What is |Sc|?

Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!

|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

What is |Sc|?Hint: The last pen is always a Good one!|Sc| = 1+ 2+ 3+ 4+ 5 = 15.

48

Question 2.5

(d) Specify the sample space in part (c) if only thenumber of pens tested is noted.

49

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

Sd = {2, 3, 4, 5, 6}

What is |Sd|?

50

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

Sd = {2, 3, 4, 5, 6}

What is |Sd|?

50

Question 2.5

Sc = {GG,GDG,DGG,GDDG, . . . ,DDDDGG}

Sd = {2, 3, 4, 5, 6}

What is |Sd|?

50

Question 2.9

The sample space of an experiment is the real line. Letthe events A and B correspond to the following subsetsof the real line: A = (−∞, r] and B = (−∞, s], where r ≤ s.Find an expression for the event C = (r, s] in terms of Aand B. Show that B = A ∪ C and A ∩ C = ∅.

51

Question 2.9

A

B

+∞−∞ r s

C = B ∩ Ac

52

Question 2.10

Use Venn diagrams to verify the set identities given inEqs. (2.2) and (2.3). You will need to use different colors ordifferent shadings to denote the various regions clearly.

53

Question 2.10

A ∪ (B ∪ C) = (A ∪ B) ∪ C

A ∩ (B ∩ C) = (A ∩ B) ∩ C

A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)

Think about it for 1 minute!

54

Question 2.10

A ∪ (B ∪ C) = (A ∪ B) ∪ C

A ∩ (B ∩ C) = (A ∩ B) ∩ C

A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)

Think about it for 1 minute!

54

Question 2.12

Show that if A ∪ B = A and A ∩ B = A then A = B

55

Question 2.12

To show that two sets A,B are equal A = B, we usuallyshow that:

1. A ⊂ B2. B ⊂ A

56

Question 2.12

To show that two sets A,B are equal A = B, we usuallyshow that:1. A ⊂ B

2. B ⊂ A

56

Question 2.12

To show that two sets A,B are equal A = B, we usuallyshow that:1. A ⊂ B2. B ⊂ A

56

Question 2.12

To show that A ⊂ B, we need to prove that for any x ∈ A,we have x ∈ B:

∀x ∈ A : A = A ∩ B =⇒ x ∈ (A ∩ B)=⇒ x ∈ B =⇒ A ⊂ B

57

Question 2.12

To show that A ⊂ B, we need to prove that for any x ∈ A,we have x ∈ B:

∀x ∈ A : A = A ∩ B =⇒ x ∈ (A ∩ B)=⇒ x ∈ B =⇒ A ⊂ B

57

Question 2.12

To show that B ⊂ A, we need to prove that for any x ∈ B,we have x ∈ A.

∀x ∈ B : x ∈ B ∪ A = A =⇒ x ∈ A=⇒ B ⊂ A

58

Question 2.12

To show that B ⊂ A, we need to prove that for any x ∈ B,we have x ∈ A.

∀x ∈ B : x ∈ B ∪ A = A =⇒ x ∈ A=⇒ B ⊂ A

58

Question 2.12

A ⊂ B and B ⊂ A =⇒ A = B

59

Questions?

59

Fun fact

We can approximate π using the concept of relativefrequency!

60

Fun fact

2

Sample points from the square “uniformly” at random.The relative frequency of points within the circle isapproximately π×12

22 = π/4

61

Fun fact

2

Sample points from the square “uniformly” at random.The relative frequency of points within the circle isapproximately π×12

22 = π/4

61

Fun fact

2

Sample points from the square “uniformly” at random.

The relative frequency of points within the circle isapproximately π×12

22 = π/4

61

Fun fact

2

Sample points from the square “uniformly” at random.The relative frequency of points within the circle isapproximately

π×1222 = π/4

61

Fun fact

2

Sample points from the square “uniformly” at random.The relative frequency of points within the circle isapproximately π×12

22 = π/4

61

Fun fact

Using Matlab after 100 iterations, sampling 10 millionpoints in each iteration:

62

Fun fact

i t e r a t i o n s = 100 ;pi_approx = ze ro s ( i t e r a t i o n s , 1) ;f o r k = 1 : i t e r a t i o n s

num_of_samples = 10000000;X = rand (num_of_samples , 1) ;Y = rand (num_of_samples , 1) ;d i s t = sq r t (X.^2 + Y.^2) ;num_samples_inside = nnz ( d i s t <= 1) ;pi_approx (k ) = 4 * num_samples_inside / ...

num_of_samples ;endformat longmean( pi_approx )p i

63

Fun fact

True value: 3.141592653589793 ...

Approximated value: 3.141594348

64

Fun fact

True value: 3.141592653589793 ...Approximated value: 3.141594348

64