Chapter S1 - Probability

7/29/2019 Chapter S1 - Probability

1/39

Students Copy

Raffles Junior CollegeH2 Mathematics

JC2 2007____________________________

__________________Chapter S1 Probability

Page 1

Chapter S1 Probability the Mathematics of Chance and Coin Tossing

PRE-REQUISITES Set Language and Notation

In particular the meaning of A, A B, A B, A B,A ',x

Venn Diagrams Permutations and Combinations

In particular, basic counting principles: addition and multiplication principle. Calculate the probability of a single event as either a fraction or a decimal Calculate the probability of simple combined events, using possibility diagrams and

tree diagrams where appropriate (in possibility diagrams outcomes will be

represented by points on a grid and in tree diagrams outcomes will be written at theend of branches and probabilities by the side of the branches)

CONTENT

0 Introduction

1 Random Experiments, Sample Space and Events Set Operations: Taking Complements, Intersections and Unions ( ', and )

Algebra of Sets: Mutually Exclusive Events

2 Probability Frequentist Approach

Theoretical Approach Axioms Properties

Subjectivist Approach

3 Conditional Probability

4. Independent Events

5. Worked Examples

0 INTRODUCTIONWhat does Randomness mean to you?

Most of us have an intuitive feeling about what it means for something to be random. Weknow that life is uncertain and speak about randomness in our everyday language. We aska weather forecaster, what is the chance or likelihoodof rain? We identify the wordrandomness to mean unpredictableor unknownwhen we ask, What is the outcome of aroll of one die ? What is the outcome of a flip of a coin ? or What does the future hold ?

A brief History An interesting question in the history of science is: Why was Probability as aMathematical theory not developed and studied until the 16th Century? In the 16th Century


2/39

Raffles Junior College H2 Mathematics JC2 2007_________________________________________


Page 2

problems in gambling made people think about probability. However, Historians have long

known that games of chance are as old as civilization. An interesting discussion can befound in the book by I. Hacking, The Emergence of Probability (Cambridge: CambridgeUniversity Press, 1975)

Among the first persons to calculate probabilities systematically was Gerolamo Cardano(1501-1576) In his book Liber de Ludo Aleae, he dealt with the special case of equallylikely outcomes. In this case, Cardano realized that the probability that an event occurs isthe ratio of the number of favourable outcomes to the total number of outcomes.

The beginning of a systematic study of Probability is attributed to a famous series of lettersbetween Pascal and Fermat. This correspondence was initiated by Pascal to consultFermat about problems he had been given by Chevalier de Mere a well-known writer,prominent figure at the court of Louis XIV and an ardent gambler !

The first problem de Mere posed was a dice problem which involved the probability ofgetting at least one six in four throws of a die. The story goes that he had been betting thatat least one six would turn up in four rolls of a die and winning too often, so he then betthat a pair of sixes would turn up in 24 rolls of a pair of dice.

The second problem concerned the determination of a fair division of the stakes in atournament (series of game) when the series is interrupted before it is completed. Thisproblem is referred to as the Problem of Points. One form of the problem is as follows:

A team plays ball such that a total of 60 points are required to win thegame and each round counts 10 points. The stakes are 10 ducats. Bysome incident they cannot finish the game and one side has 50 points andthe other 20 points. One wants to know what share of the prize money

belongs to each side.

It was only as recently as in 1933 that a universally accepted axiomization (See Section 2for the Axioms) of the theory was proposed by A. N. Kolmogorov. Gradually, thesedevelopments branched out far beyond the games of chance to encompass many otherfields associated with chance occurrences, such as politics, business, insurance, weatherforecasting, and scientific research. Some applications are:

1. According to quantum theory of physics, we use probabilities to describe theorganization of subatomic particles, atoms in crystals and gas molecules.2. Probabilistic understanding of how genes are transmitted from parents to offspringis the basis of models of inheritance.3. The model of the AIDS epidemic is based on probability arguments.4. Banks apply probability to assess the risk of granting credits to different applicants.

With additional computing power, simulation is now employed to model these differentkinds of chance behaviour and answer questions which are otherwise beyond the reach oftheoretical calculations.

To create realistic Mathematical models, make informed predictions and forecasts anunderstanding of basic probability theory is essential.


3/39



Page 3

The concepts in this first section should be familiar to you.

1. RANDOM EXPERMENTS, SAMPLE SPACE AND EVENTSIn this section we discuss the concept of an experiment, the sample space and events.

Our goal in Section 2 Probability will be to assign probabilities to events so we first needto understand how to Mathematically describe what events are, and how to deal with them.

A random experiment is any process of observation or measurement of aphenomenon which has multiple possibleoutcomes. We use the word outcome tomean a result (instrument reading etc) of the experiment and this result may or maynot be numerical. An outcome may consist of more than one item of (qualitativeand/or quantitative) information. The outcome prior to the experiment cannot bepredicted with certainty. However, we assume that we are aware of all possible

outcomes.

For each experiment, the set consisting of all possible outcomes is called thesample space, of the experiment. Each outcome or element in the samplespace is called a sample point. Sample space can be finite or infinite sets.

An event A is a subset of the sample space and is typically denoted by upper caseletters A, B, C etc. i.e.A . In other words, an event A is a set consisting of

possible outcomes of the experiment.

An event that consist of a single outcome is referred to as an elementary or simpleevent. An event that consist of more than one outcome is called a compoundevent. We say that an event A occurs when the outcome of the experiment is any

element of the set A.

The set is called the sure or certain event since some member of mustoccur. The empty set or { } is called the impossible event.

Example 1An experiment consists of tossing a fair coin.What are the likely outcomes ? What events are we interested in ?The sample space ={Heads, Tails}Let H={Heads} then H is the event of obtaining a heads with the toss of 1 coin.

A coin is flipped and Heads obtained.So we say, event H has occurred.

Stop and Think: What would be a simple representation for sample space for 100 flipsof a coin ?How big would the sample size be?Is it possible for the coin to land and balance on an edge ?Should we include it into our sample space?

Comment [S1]: A random variable is anumericalvalue associate with the

outcome of an experiment.

Comment [S2]: Discrete (i.e.countable) or Continuous (uncountable).

Comment [S3]: If we code H by 1and T by 0 then we can represent thesample space as all binary sequences with

100 digits.

Hence, sample size is 1002

Comment [S4]: We could include it inthe sample space. We will then have tothink about the probability assigned to

such an event. Likely, to set P(E)=0.


4/39



Page 4

Example 2

An experiment consists of rolling a die.The sample space {{{{ }}}}1,2,3,4,5,6 = = = = .

What are some events of interest ?

Let E be the event of obtaining an even number.Then translating into set language E={2,4,6}

Let F={1,2}, then F is the event obtaining a number less than 3.

Suppose the outcome of the roll of a die is 2, 2 E and 2 F then we say that theevent E has occurred AND the event F has occurred

Suppose the outcome of the experiment is a 6, 6 E but 6 F then we say that theevent E has occurred BUT the event F has NOT occurred.

Stop and Think: Let G={7}. Is G an event ?Is it possible to define our sample space orZ R = = = = ?

We can form a 2-way table (or possibility diagram) to list down all the outcomes ofexperiments in the following instances(i) Experiments which records 2 measurement(ii) Experiment that has 2 sub-stages.

Example 3An experiment consists of associating lung disease and smoking. A man aged 60 israndomly selected and asked whether he is a smoker and whether he has lung disease.

The sample space can be written as entries in the following 2-way table.Smoker Non-Smoker

Has LungDisease

Smoker & Has lung Disease Non-Smoker & Has lung Disease

Does NOT haveLung Disease

Smoker & Does NOT havelung Disease

Non-Smoker & Does NOThave lung Disease

Example 4A student is asked for the month of the year and the day of the week on which her birthdayfalls. The sample space can be written as the entries in table as above.Alternatively, we can represent the sample space as points on the grid

2 {( , ) : 1, 2,...,12 ; 1,2,...,7}x y x y = = = = = = = = = = = =

where we havecoded

the monthsand the days of the weekin terms of numbers.

Let2E {( , ) : 7}x y x y= + == + == + == + =

Then, in terms of the actual outcomes

2E {(1,6),(2,5),(3,4),(4,3),(5, 2),(6,1)}====

Day2

7 * * * * * * * * * * * *

6 * * * * * * * * * * * *

5 * * * * * * * * * * * *

4 * * * * * * * * * * * *

3 * * * * * * * * * * * *

2 * * * * * * * * * * * *

1 * * * * * * * * * * * *

1 2 3 4 5 6 7 8 9 10 11 12 Month

2E

F

1 2

3 4

5 6

E

Comment [S5]: No, {7} does not lie inour sample space.

Comment [S6]: Yes, as a MathematicalModel.


5/39



Page 5

We can use a Tree diagram to represent all the outcomes of an experiment which has 2

or more stages. A path through the tree corresponds to a possible outcome of theexperiment.

Example 5A bag contains 7 white and 3 black balls.A ball is picked randomly three times, each ball replaced after it is drawn.Describe the outcomes of the experiment.Describe the event A that at least 2 white balls are selected

Probability tree:1s ball 2n ball 3r ball Outcomes

W WWW

B WWB

W WBW

B WBB

W BWW

B BWB

W BBW

B BBB

Using our tree diagram A={WWW,WWB,WBW,BWW}

Stop and Think: Are all the outcomes equally likely ?Compare the 2 outcomes WWW and BBB.

So far our examples all have a finite sample space.The next example illustrates a sample space is infinite.

Example 6An experiment consists of testing the lifetime of a batch of batteries when used on an ipod.The time in which the device fails is recorded.

The sample space 0++++ = = = = R

What are some events of interest ?Let E be the event the device failsby time 30.

Then {{{{ }}}} [[[[ ]]]]E : 0 30 0,30x x= == == == =R

Let {{{{ }}}} [[[[ ]]]]F : 20 30 20,30x x= == == == =R then F is the event the device stops between time 20 and 30(inclusive).

Observe that F E . This means that if the event F occurs then E occurs as well.

Stop and Think: Why did we include the point {0} from the sample space ?Is the point {30} an event ? Can you interpret what this means ?

W

B

W

B

W

B

0 20 30

F

E

Comment [S7]: 1)events along thesame branch are related by and

2)events on different branches are relatedby or

Comment [S8]: NO ! Intuitively itshould be clear that WWW is more likely

than BBB since there are more White ballsthan Black.

Comment [S9]: Indicates defectivebatteries.

Comment [S10]: Yes. Event ipod failsat exact time 30.


6/39



Page 6

Exercise 1

What is an appropriate sample space for the lifetime of a human ?Who might be interested in such an experiment ?What are possible events of interest ?

Historical Remark: UrnsIn the seventeenth and eighteenth centuries lotteries often involved the drawing of slipsfrom urns and voting was often a matter of putting slips or balls into urns. It was therefore

not unnatural for the numerous Mathematicians of the day to model births, marriages,deaths, fluids, gases and so on using urns containing balls of various hues.

Exercise 2An urn (box) contains 6 marbles: 2 red, 3 green and 1 blue.

Consider an experiment that consists of taking 2 marbles one at a time from the urnwithout replacement.Describe the sample space.Describe the possible outcomes of the event that one of the marbles is blue.

The next exercise applies ideas from the earlier Chapter on Permutations andCombinations

Exercise 3An experiment consists of selecting a committee of 3 from a group of 3 men and 2 women.Describe the sample space.Describe the event A that the committee has at most one woman.

Stop and Think: Given that sample space has a finite number of elements{1,2,3,,n}. How many possible events can be formed ? Describethese events

Comment [S11]: This is different fromselecting 2 marbles at 1 time. Unless we,take one with each hand and read off fromleft to right.

Comment [S12]: Sample size is n.The number of possible events that

can be formed are 2n

.For each event we can either includeor exclude each of the n possibleoutcomes.


7/39



Page 7

The family of subsets of called events.

Given a collection of sets from which we call events, what other kinds of sets canbe constructed? What set operations do you know ?

Also, the Mathematical notion of an event as a set should correspond to our intuitive(everyday) understanding.

If A is an event and hence we are able to ask whether A occurs, sensibly it shouldmean that we can ask whether A does not occur. That is, if A is an event then thecomplement of A , A ' must also be an event.

Furthermore. if we can observe whether the individual events A and B occur, surelythe event that A and B both occur should also be regarded as an event.

What does the event A B mean ?

Set Operations Shade appropriate region1. The complement of the event A, denoted by 'A , is

the event that Adoes not occur.

2. The intersection of the two events A and B, denotedby A B , is the event that both A and B occursimultaneously. Alternatively, it can be regarded as theevent that A and B follow in stages or sequence.

3. The union of the two events A and B, denoted byA B , is the event that at least oneof the events occurs.

In English when we use the phrase either or wetypically mean one but not the other. To say that the eventA B occurs means A occurs only ORB occurs only ORboth A and B occur.

Exercise 4 :Given 2 events A and B(i) What is the corresponding set such that A occursONLY (that is A occurs but B does not occur) ?

(ii) What is the corresponding set such that neither Anor B occurs?

A

A '

A

B

A B

A B

A B


8/39



Page 8

Example 7

To illustrate the above definitions, consider rolling a die.The outcome of the roll cannot be predicted exactly.However, we do know that the possible outcome is one of the numbers: 1, 2, 3, 4, 5 or 6.Hence the sample space {1, 2,3,4,5,6} = . Each one of these elements is a sample point.

Let A be the event that an even number occurs. Then {2,4,6}A = .

We say that A occurs when any one of the three numbers 2, 4 or 6, occurs.Let Bbe the event that a prime number occurs. Then {2,3,5}B = .

Find (i) A B , A B and 'A (ii) the event the outcome of the experiment is an even number that is NOT

prime.Solution:

(i) {2}A B = corresponding to the event outcome is an even number AND a

prime number.{2,3,4,5,6}A B = , corresponding to the event that the outcome is either an

even number OR prime number OR both.' {1,3,5}A = corresponding to the event that the outcome in NOT an even

number.

(ii) A B'={4,6} corresponding to the event that the outcome is an even number

that is NOT prime.

Exercise 52 Dice are thrown.

Let E be the event the sum of the outcome of the dice is even.Let F be the event that at least one of the dice lands on 1Let G be the event that the outcome of both dice is the same.

Complete the Possibility diagram:First die

1 2 3 4 5 6

1 (1,1) (2,1)

2 (1,2) (2,2)

3

4

5

6

Describe the events F , E ' , E F , E ' F , F G and E F G

+

Secondd

ie


9/39



Page 9

Stop and Think:

Given 3 events A, B and C, what is the event using set notation corresponding to(a) Exactly one event occurs ? and (b) Exactly two events occur ?A Venn Diagram might help.

See Appendix to Section 1 for more on Set Theory.

Mutually Exclusive events : Two events A & B are mutually exclusive if either event Aor the event B can occur, but not both at the same time.Alternatively, we can interpret mutually exclusive events as saying that if the event Aoccurs it implies that B does not occur and vice-versa.

Example 8(a)A coin is tossed.

H=event Heads is obtained.T=event Tails is obtained.(Simple) Events H and T are mutually exclusive.If H occurs then T cannot occur.

Example 8(b)A die is rolled

E=event number is even.O=event number is odd.Events E and O are mutually exclusive.A number cannot be both in E and O

Example 9Randomly select a student from class.A=event selected student regularly sleeps during Mathematics lessonsB=event selected student scores As in all his Mathematics tests and examinations.Are events A and B mutually exclusive ? Discuss.

In set language, mutually exclusive events A & B mean that the sets A and B are disjoint.

Notice A and B do NOT overlap !

The following are equivalent ways of describingmutually exclusive events.(i) A B = by definition.

(ii) ' and 'A B B A

(iii) ' and 'A B B B A A = =

Convince yourself using the Venn Diagram.

Mutually exclusive event are important in probability because the probability of atleast one occurring is the sum of the probabilities that they occur individually.

The following are some important mutually exclusive events.

Example 10A and A' are mutually exclusive events.

A A' =

In a finite sample space, ( ) ( )n( ) n A n A' = +

Hence, ( ) ( )n A' =n( ) n A

AB

A

A '

Comment [S13]:(a) ( ) ( ) ( )' ' ' ' ' 'A B C A B C A B C

(b) ( ) ( ) ( )' ' 'A B C A B C A B C

Comment [S14]:

' 'A B B A Special Case A=B, B and B are disjoint

Comment [S15]:

' 'A B B B A A = =

Comment [S16]: Later we will

discover that ( ) ( )P A' =P( ) P A


10/39



Page 10

Example 11

A B and 'A B are mutually exclusive events.

( ) ( )A A B A B'=

In a finite sample space, ( ) ( )n(A) n A B n A B'= +

Exercise 6(i) Show that A B' and B are mutually exclusive events using a Venn Diagram.

(ii) Show that , ',A B A B 'A B and ( )' ' A B 'A B = are mutually exclusive events

using a Venn Diagram.

Exercise 7

In (a), (b), (c) & (d) below, n( )=8, n(A)=3 and n(B)=4 where n(E) denotes the number ofoutcomes in event E. We use a * to represent an outcome of the experiment.

Find (i) n(A B) (ii) n(A) (iii) n(B) (iv) n(A B) (v) ( )n A' B'

Hence, verify the following 2 set properties:

(i) ( ) ( )n(A B)=n A +n(B) n A B

(ii) ( ) ( )n A' B' n( ) n A B =

(a) (b)

(c) (d)

* * * * * * * * *

* *

*

* * * *

A B A B

* * ** * *

* *

** * *

*

* * *

A B AB

AB


11/39



Page 11

Exercise 8100 000 people are asked whether they read Today (Paper I), The Straits Time (PaperII) and/or The New Paper (Paper III)

Paper I: 10 000 Paper I & II : 8 000 Paper I, II & III: 1000Paper II: 30 000 Paper ! & III : 2 000Paper III: 5 000 Paper II & III : 4 000

(a) Find the number reading only 1 newspaper(b) How many people read at least 2 newspapers ?(c) How many dont read newspapers at all?

Paper I and III are afternoon papers and Paper II is a morning paper.(d) How many people read at least one morning and one afternoon paper?(e) How many people read one morning and exactly one afternoon paper ?

Now that we know all about events, we need a way to assign some number that reflectsthe likelihood of the event occurring.2 PROBABILITYProbability is a measure of the likelihood of an event occurring and is measurednumerically on a scale from 0 to 1.An event that is more likely to occur has a higher probability than one that is less likely tooccur.An impossible event is deemed to have a probability of zero and a sure or certain eventhas a probability of one.The probability of an event E will be denoted by P(E) .

We can regard P(E) as a function defined over sets E .

It is important to think about how probabilities are determined in practice. One

simple way is by symmetry. For the case of a toss of a coin, we do not see anyphysical differences between the two sides that should affect the chance of oneside turning up. Similarly with an ordinary die.

We have to be careful though. It has been observed that in a given population theproportion of newborn children who are boys is 0.513. Hence, it may be moreappropriate to assign a probability of 0.513 that the sex of a newborn is male. Thisis an example where we use statistical observations and a frequency notion todetermine probabilities.

Set constructions are important because it is typically the case that complicated eventsdescribed in English can be broken down into simpler event using these constructions.


12/39



Page 12

Some situations cannot be repeated a number of times such as the outcome of this

years Champion League Soccer tournament. However, a personal belief orprobability that your favourite team (e.g. Manchester United) will emerge asChampions can be determined by the kind of bet you are willing to make.

There are basically three approaches to assign a probability to a given event.These approaches are very different in principle and in practice.

2.1 Frequency/Empirical Approach

Suppose we were interested in determining the probability of obtaining a head in 1 toss ofa coin. Could we conduct an experiment to help us?

We could flip the coin a number of times say 100 or 1000 or 10000 and look at the

proportion of times that heads actually occurs. We could then estimate the probability bythe ratio obtained.

Intuitively, it is plausible to believe that the long run proportionof heads obtained will beequal to the Probability of Head, that is .

In the previous example we would have other reasons to believe that P(H)=1/2 based onthe reason that the occurrence of Heads should be as equally likely as Tails.For the next example we wouldnt really have any idea what probability to assign.

Example 12Consider the experiment of dropping a thumb tack from a height and recording whether itpoints-up or points-down.

Let E be the event of interest thumb-tack points-up.How would you deduce the Probability that a thumb-tack will eventually lie facing up ?Well, we toss a thumb-tack repeatedly, and keep track of the number of throws ( n) and the

number of times the thumb-tack points up n(E). Then calculate the ratio( )n E

n.

This will form an estimate of the probability.

As nincreases, we can expect this ratio( )

limn

n E

nto stabilize.

The probability P(E) is the limiting value of this ratio as ntends to infinity.

In this case, the probability is derived from empirical evidence, i.e. a Statistical approach.

In the long run, the ratio ( )n En

tends to the actual probability of the event P(E).

This ratio is sometimes referred to as the relative frequency or proportion.

Stop and ThinkDoes it seem to be a priorievident that in any experiment with event of interest E, the ratio

( )n E

nshould converge to some limiting value?

Comment [S19]: A powerful modernapproach to solving complicate questions

in probability is through simulation.

Comment [S20]: Using the axioms ofprobability we will show that thisproportion does converge a result called

the law of large numbers.


13/39



Page 13

2.2 Theoretical Approach

Consider an experiment whose sample space is .For each event E of we assume that there is a number P(E) referred to as theprobability of E which measures the relative likelihood that the event E occurs.

The probabilities that we have assigned for each event E in the sample space shouldsatisfy some intuitive notions.

The probability of an event is measured along a scale from 0 and 1.

Axiom 1 ( )0 1P E

We said at the onset that the set of all possible outcomes is known.The probability that an outcome will be a point in the sample space is 1

Axiom 2 ( ) 1P =

For mutually exclusive events 1 2 3, , ,...E E E the probability that at least one of them occurs is

just the sum of their respective probabilities.

Axiom 3 ( ) ( ) ( ) ( )i 1 2 3 1 2 3if E when i j then ... ...jE P E E E P E P E P E = = + + +

Axiom 3 says that Probability P( ) is a continuous set function

Stop and Think: Let E be some event of interest of an experimentSuppose the experiment is repeated n timesLet n(E) = number of time the event E occurs

Definen(E)

f(E)

n

= . Show that f(E) satisfies the Axioms 1,2 & 3.

Sample Space having equally likely OutcomesFor many experiments it will be quite reasonable to assume that all outcomes are equallylikely to occur. For example, flipping a coin or rolling a die or drawing a card from a deck ofcards, etc.

Consider an experiment whose sample space {1, 2,3,...., }n =

Assume that P({1})=P({2})=P({3})==P({n})

Hence by Axiom 2 and Axiom 3 P({1})=P({2})=P({3})==P({n})=1/n

By Axiom 3 again, for any event E,number of points in E

( )

number of points in

P E =

In the case of a finite sample space whereby all the sample points are equally likely to

occur, the probability of an event E is given byn( )

P( )n( )

EE =

, where n( )E denote the

number of elements of the set E.

Comment [S21]:(i) n( )

n( )=n, so f ( ) 1n

= =

(ii) n( )0 n( ) n, so 0 f ( ) 1

n

EE E =

(iii)1 2if E E =

( ) ( ) ( )1 2 1 2

n E E n E n E = +

( ) ( ) ( )1 2 1 2

f E E f E f E = +

Comment [S22]: Flipping a coin,Tossing a die and Drawing a card are allexamples of experiments with outcomes

that are equally likely.


14/39



Page 14

Consider the experiment of rolling a die again.

If the die is fair/unbiased, intuitively we know that it is equally likely for the outcome to beany of the 6 sample points {1,2,3,4,5,6}. That is to say that each of the 6 faces has anequal chance of turning up.

We can therefore argue that the probability of getting a "4" is 1 out of 6, or1

6

Stop and Think : Can we deduce this fact by using the Axioms of Probability ?

Example 131000 students were polled at Pioneer-Thinker-Leader School on whether they readprinted News (Newspaper) and/or watch the News on Television.200 replied that they read the news only.100 replied that they follow the news over the television only.100 replied that they do both.

A student is randomly selected from the 1000 students polled(i) What is the probability that the student reads the Newspaper ?(ii) What is the probability that the student selected neither reads the newspaper nor

watches the news on TV ?(a)(i) P(reads newspapers only)

= 200/1000=0.2

(ii) P( doesnt follow news in newspaper or TV)=600/1000=0.6

Stop and Think: A student is randomly selected from the 300 students who readsnewspapers. In this situation, what do you think is the probability thatthe student does not watch the news on TV?

Assigning probability based on the assumption that each outcome is as likely to occur asany other is using the notion of symmetry.

The following properties of probability can be easily deduced if the sample space isfinite (the set has finite number of elements) and the outcomes are equally likely.Recall first the following set properties:

Let ( )An denote the number of elements in the set A.

1. ( ) 0n =

2. For disjoint sets1 2E and E , ( ) ( ) ( )1 2 1 2n E E n E n E = +

3. ( )( ') ( ) where denotes the universal set.n E n n E =

4. ( ) ( )( ) ( )n A B n A n B n A B = + and ( ) ( )' ' ( )n A B n n A B =

5. ( ) ( )( ) 'n A n A B n A B= +

Newspaper TV

200 100 100

600

Comment [s23]: Unless otherwisestated, we may assume that a givendie is fair.

Comment [S24]: We know thatP({1})=P({2})=P({3})=P({4})=P({5})=P({6})

Now use Axiom 2, ( )P 1 = By Axiom 3

( ) ( )P P {1} {2} {3} {4} {5} {6} =

( ) ( ) ( ) ( ) ( ) ( ) ( )

( )

( )

P P {1} P {2} P {3} P {4} P {5} P {6}

1 6P {1}

1P {1}

6

= + + + + +

=

=

Comment [S25]: Prob =2/3


15/39



Page 15

Now we have the analogous Properties of Probability

1. ( ) 0P =

We already know that the sure event has probability 1 of occurring by Assumption.Property 1 says that the impossible event has No probability (0 probability) ofoccurring.

2. For mutually exclusive events 1 2E and E , ( ) ( ) ( )1 2 1 2P E E P E P E = +

3. The complement of an event E has Probability ( ') 1 ( )P E P E =

Property 3 states that the probability that an event does not occur is 1 minus theprobability that it does occur.

4. Given 2 events A and B, the probability that at least one of the 2 events occurs is,

( ) ( )( ) ( )P A B P A P B P A B = +

Property 4 is called the Addition Rule of Probability !

The probability that neither A nor B occurs is given by

( ) ( )' ' 1P A B P A B =

5. ( ) ( )( ) 'P A P A B P A B= +

Given 2 events A and B, the probability that A occurs only (but B does not),

( ) ( )' ( )P A B P A P A B =

Stop and Think :What if the sample space is NOT finite ? Can we still prove that the properties hold ?

Go back to first principles and use our Axioms !Example 14

A card is drawn at random from an ordinary pack of 52 playing cards.Find the probability that the card is(i) not a 7;(ii) a Club or a Diamond(iii) a Club or a King.(iv) neither an Ace nor a Heart.

Comment [S26]: The form

( ) ( )( ) 'P A P A B P A B= + is very

useful as well.

Comment [S27]: See appendix toSection 2.


16/39



Page 16

Exercise 9

Consider a Toss of 2 coins.Let A=event 1st coin falls heads={(H,T),(H,H)}Let B=event 2nd coin falls heads={(H,H),(T,H)}

(i) Find and interpret ( )P A B

(ii) Find and interpret ( )'P A B

(iii) Express the event C= neither the 1st nor 2nd tosses yields a Heads in terms ofevents A and B, Hence, find P(C).

(iv) Express the event D, obtain 1 Heads in 2 tosses of a coin in terms of events A andB. Hence, find P(D).

Solution:

(iv) P(1H in 2 tosses of a coin)= ( )P 'A B + ( )P 'A B = ( ) ( )1 1 1

P HT P TH4 4 2

+ = + =

In the previous exercise the 22 4= outcomes of the experiment HT,TH,HH & TT are equallylikely. Observe from part (iv) that the number of heads possible in 2 tosses of a coin are

0,1 or 2. So a different experiment could consist of simply counting the number of heads in2 tosses of a coin. Note here that the number of heads possible in 2 tosses of a coin areNOT equally likely.

Stop and Think: Consider a toss of 3 coins.An experiment consists of counting the number of heads.What is the probability of 0 Heads, 1 Heads, 2 Heads and 3 Heads ?

Example 15In a probability experiment, events A and B are such that

P( A' )=0.75, P ( )1

A B12

= and P( A' B' )=0.25

Find

(i) P( A B ) (ii) P( A' B ) (iii) P( B')

Solution:(i) P( A B )=1 P( A' B' )=0.75(ii) P( A' B )=P( A' ) P( A' B' )=0.5

(iii) P(B)= P( A' B ) + P ( )A B =7

12OR P(B)= P( A B ) P(A)+ P ( )A B =

7

12

P( B')=1-P(B) =5

12

Comment [S28]: 1/8, 3/8, 3/8, 1/8


17/39



Page 17

In the next exercise, the outcomes of the experiment are NOT equally likely !

However, we can determine the probabilities of each branch using the idea of equallylikely outcomes.

Example 16An urn contains 10 marbles: 4 red and 6 green.

Consider an experiment that consists of taking 2 marbles one at a time from the urnwithout replacement.

Draw a Probability Tree diagram. Indicate clearly all probabilities.

Solution:.

Exercise 10

If B A show that ( ) ( )' ( )P A B P A P B =

Deduce that ( ) ( )P B P A

Exercise 11Deduce the following properties about mutually exclusive events A & B(i) ( ) 0P A B =

(ii) ( ) ( ')P A P B and ( ) ( ')P B P A

If P(A) =1/3 and P(B)=1/4 can A and B be disjoint? Why?


18/39



Page 18

Null Events: An event E such that P( ) 0E = is called a nullevent.

Null events should not be confused with the impossible event .An impossible event IS a null event since P( ) 0 =

Consider the toss of a coin and suppose we include the 3 possibilities outcomes asobtaining a head or tail or landing on an edge.

{ , , }Heads Tails Edge = It is reasonable to assign P{Edge}=0

A more technical example would be to consider an experiment of throwing a dart ata board then the P{darts strikes any given point of the target at which it is thrown}=0

Consider the roll of one die with sample space {1, 2,3,4,5,6} =

It would seem intuitive that P({7})=0However, {7} is NOT even an event so it is meaningless to talk about assigning a

probability to this event.

2.3 Subjectivist Approach

We use the term probability in many ways. For instance, people make thestatement that it is 90% certain that Shakespeare actually wrote Hamlet. A simpleinterpretation here is that the probabilities are a measure of the individuals belief inthe statements that she is making.

It seems logical to suppose that a measure of belief should satisfy all axioms ofprobability.

For example, if we are 70% certain that Shakespeare wrote Julius Caesar AND10% certain that it was actually Marlowe then we are 80% sure it was eitherShakespeare or Marlowe.

See Appendix for more details.

The next section introduces one of the most important ideas in Probability.3. CONDITIONAL PROBABILITY

In this section we will answer the following questions:1. Suppose we have assigned probabilities to all events A in a sample space andthen learn that an event B has occurred. How should we change the probability of theevent that A occurs given the additional information that B has occurred?

2. Suppose we know the incidence rate within a population of a given event (e.g.disease). We may be interested in the incidence of this same characteristic (e.g. disease)within a sub-population. (e.g. teenagers)

Example 17

An experiment consists of rolling a die.The sample space {{{{ }}}}1,2,3,4,5,6 = = = = .

Consider the following 2 events:A=event outcome is a 6={6}B=event outcome is even ={2,4,6}

The experimenter asks, Assuming a fair die, what is the probabilitythat event A occurs?

B

1 2

3 4

5 6

A


19/39



Page 19

Solution: Since all outcomes are equally likely, P(A)

1

6=

A die is rolled a 2n time and without revealing the actual outcome an observer says eventB has occurred. (the given information)With this additional information what is the probability that event A occurs?

Solution: Since the event B has occurred, we know that the possibleoutcomes of the experiment are {2,4,6}The sample space has been reduced or restricted to this set of outcomes.In the absence of any other information and regarding the outcomes {2,4,6}as equally likely, the probability that the outcome is a 6 with this additional

information is1

3.

Computing P(A|B)If the event B has occurred, then in order for A to occur, itmeans that necessarily the actual occurrence must bea point of both A and B.( that is A B )

Now B is the new reduced sample space, hence theprobability that A B occurs (relative to B) is equal to theRatio of the Probability of A B to the Probability of B.

In other words,

If A and Bare 2 events such that P( ) 0B .

The probability of A occurring given that Bhas already occurred, is given byP(

P( | )P( )

A BA B

B

)= .

Looking back at our first exampleP( ({6}) 1/ 6 1

P( | )P( ) ({2, 4,6}) 3 / 6 3

A B PA B

B P

)= = = =

Example 18Suppose two fair coins (labeled 1 and 2) are tossed and you are asked to place a bet onthe outcome of the experiment. The sample space is = {HH, HT, TH, TT}, and each ofthe four sample points has a equal chance of occurring (i.e. probability of 0.25 ofoccurring)

Therefore, if you bet on the event A that both the first and second tosses have the

same outcome i.e. HH or TT, your chance of getting it right is 0.25+0.25=0.5.

However, if you are told that one of the coins lands H (event B) after you havebetted on A what is the probability that you are right? Is it still 0.5? Has theadditional information affected your evaluation of your chances?

Explanation:If you know that the event B that one of the coins shows H, your perception of the samplespace is no longer {HH, HT, TH, TT} but {HT, TH, HH}.

A B

A B

B = B

2

4

6 A

B is known as the

reduced sample space.


20/39



Page 20

In the latter case, each of the three sample points has an equal chance of occurring.

Also, for you to win the bet, the only favourable outcome is HH as knowledge of event Bhas ruled out TT

It is clear that the probability of HH under this condition is now1

3.

Hence, the probability that we win our bet has dropped to1

3.

This probability is known as the conditional probability of HH or TT giventhat there isat least one H. The event that there is at least one 'H' is the condition.

In our example, A = {HH,TT}, B= {HT, TH, HH} and {HH}A B =

We have

1

P( ) 4A B = and

3

P( ) 4B = . Hence,

1P( 14

P( | ) 3P( ) 34

A B

A B B

)

= = =

Stop and Think: What happens if you are told event C that the first coinlands H?Will the evaluation change? Try reasoning it out for yourself!

One more example to illustrate further the idea of the reduced sample space.Example 19Actuaries use Life-Tablesto estimate the probability using historical records that someonecurrently aged x lives at least another t years.In a given Life-Table one finds that in a population of 100 000 females (living say in a Cityin Country X), 89 835 live past the age of 60, while 57 062 live past the age of 80.

An individual is randomly selected from the population of 100 000 women.

(i) Find the probability that the selected individual lives past the age of 80.

An individual is randomly selected among the sub-population of 89 835 women who livepast the age of 60.(ii) Find the probability that the selected individual lives past the age of 80.

Stop and Think:An individual is chosen at random from the population of women in Country X(iii) Given that a woman is currently aged 60 estimate the probability that she lives pastthe age of 80?

We consider E to be the new sample space, and note that F is a subset of ENote : women who live past the age of 80 years must have lived past the age 60 !

(ii)( ) ( ) 57 062

P( | ) 0.6352( ) ( ) 89 835

n F E n F F E

n E n E

= = = =

SolutionThe original sample space can be thought of as theset of all 100 000 females.

Let E and F correspond to the subset of the samplespace consisting of all women who live past the ageof 60 and age 80 respectively.

(i)( ) 57 062

P( ) 0.57062( ) 100 000

n FF

n= = =

n(F) 57062=

n(E) 89835=

n() 100000====

Comment [S29]: 1P( | )2

A C =

Given first coin is H, there is nowone favourable outcome HH out of 2possible outcome {HT,HH}

Comment [S30]: We estimate thisprobability as 0.6352


21/39



Page 21

Example 20

It is known that there is a link between smoking and lung disease.There is concern about smoking among young people.In a survey of 100 000 individual the proportion of smokers is estimated at about 0.15.3000 smokers are between the age of 15 to 25.It is estimated that the 15 to 25 year olds form 25% of those surveyed.

Randomly select an individual from the study.Let S=event that the individual is a smokerLet A=event that the individual is between age of 15 to 25.(a) Find the following 3 probabilities: P(S), P(S|A) and P(A|S)(b) Find P(S|A) and P(A|S)

Solution:

Smoker Non Smoker TotalAge 15-25 3 000 25 000Not Age 15-25Total 15 000 100 000

We can rewriteP(

P( | )P( )

A BB A

A

)= as P( ) P( )P( | )A B A B A =

The Multiplication Rule:

P( ) P( )P( | )A B A B A =

Now we will consider some examples using Probability Trees.Observe that from the Multiplication Rule :

(i) P( ) P( )P( | )A B A B A =

(ii) P( ') P( )P( ' | )A B A B A =

(iii) P( ' ) P( ')P( | ')A B A B A =

(iv) P( ' ') P( ')P( ' | ')A B A B A =

Note: These are the probabilities along each branch of the tree.

Example 21

Two events A and Bare such that1

P( )3

A = ,1

P( | )4

B A = and4

P( ' | ' )5

B A = .

By drawing a tree diagram, find(i) ( ' | )P B A , (ii) )( BAP , (iii) )(BP , (iv) )( BAP .

A

B'

B

B( | )P B A

( ' | )P B A

( | ')P B A

( ' | ')P B A

( )P A

( ')P A

B'

A'

P( )A B

P( ')A B

P( ' )A B

P( ' ')A B


22/39



Page 22

Solution:

Given1

P( )3

A = ,1

P( | )4

B A = ,4

P( ' | ')5

B A = , construct the probability tree:

From the tree diagram,

(i)3

P( ' )4

B A =

(ii)1 1 1

P( )3 4 12

A B = =

(iii) P( ) P( ) P( ' )B A B A B= + 1 1 2 1 13

3 4 3 5 60

= + =

(iv) P( ) P( ) P( ) P( )A B A B A B = + 1 13 1 7

3 60 12 15= + =

Stop and Think: We know P( ) P( )P( | )A B A B A = . Now, using the probability tree

deduce and interpret the following product P( )P( | )P( | )A B A C A B

Exercise 12Anne is undecided whether to take Mathematics OR Chemistry.Although she prefers Mathematics, Anne estimates that her probability of receiving an Agrade during A level would be 0.5 in Chemistry whereas it would only be 0.25 inMathematics.Anne decides to base her decision on a roll of a die with outcomes {1,2,3,4} meaning adecision to take Mathematics and {5,6} meaning a decision to take Chemistry.

(i) Find the probability that she gets an A in Mathematics ?

(ii) Find the probability that she gets an A in Chemistry ?

(iii) Find the probability that she gets an A in whatever course she takes ?

Example 22The probability that a golfer strikes the ball on to the green if it is windy as he strikes theball is 0.4, and the corresponding probability if it is not windy as he strikes the ball is 0.7.The probability that the wind will blow as he strikes the ball is 0.3.

Find the probability that(i) he strikes the ball on to the green;

(ii) it was not windy, given that he does not strike the ball on to the green.

A

B'

B

B14

34

15

45

13

23

B

A

Comment [S31]: This is simply

P( )A B C


23/39



Page 23

Stop and Think:A doctor gives a patient a test for cancer. Before the results of the test, the only evidencethat the doctor has to go on is that 1 woman in 1000 has this cancer.Experience has shown that in 99% of the cases in which cancer is present, the test ispositive; and in 95% of the cases in which it is not present, it is negative.

If the test turns out to be positive, what probability should the doctor assign to the eventthat cancer is present ?

Intuition would lead us to believe that the answer should be at least 0.5.Is this correct ?Suppose it is known that this cancer is found in 50 women in 1000.What probability should the doctor assign to the event that cancer is present given apositive test?A patient who tests positive is tested again.What is the probability of disease given a 2nd positive test if the incidence rate of cancer is50 women in 1000 ?

One more important remark. Under the reduced sample space, Conditional Probabilitiessatisfy the axioms of probability in Section 2 and hence IS a probability.

Therefore all the properties we saw earlier also hold, such as:

(i) ( ) ( )| ( | ) ( | ) |P A B C P A C P B C P A B C = +

(ii) ( ) ( )' | ( | ) |P A B C P A C P A B C =

Comment [S32]: 99/5094

Comment [S33]: 1/2

Comment [S34]: 99/104


24/39



Page 24

4 INDEPENDENT EVENTS

We saw in the previous section that P(A|B) the conditional probability is not generallyequal to P(A). In other words, knowing that an event B has occurred generally changes thechances or probability of A occurring.

In the special case where P(A|B)=P(A) we say that A is independent of B.

A and B are independent if(i) P(B)>0 and P(A)>0

And (ii) P( | ) P( )A B A= or P( | ) P( )B A B=

In other words, two events are independent if the occurrence or non-occurrence of oneevent has no influence on the probability of the occurrence or non-occurrence of the

other. (note: the influence here is on the chance mechanism.)

2 events that are not independent are said to be dependent.

For example, if a coin is tossed and a die is rolled, which side the coin lands will in no wayaffect the outcome of the die (and vice-versa). Hence, the outcome of a single toss of acoin and the outcome of a single roll of a die are independent events.

From the definition, if A and Bare 2 independent events, we haveP( | ) P( )A B A= and P( | ) P( )B A B=

P(P( )

P( )

A BA

B

) =

P( P( ) P( )A B A B ) =

Independence plays an important role in probability and statistics. It is good if you can getan intuitive feeling of this concept, but that can be difficult sometimes.

A and Bare independent events, is equivalent toP( P( ) P( )A B A B ) = .

Example 23Suppose that we toss 2 fair dice.Then the outcome of the first toss should in no way cause any outcome of the second tossto be more or less likely.(i) If A=event first toss is a 4.

If 1B =event second toss is a 3.

Then P( 1B |A)=P( 1B )

(ii) If2B =event second toss is a 2,

Then obviously1

B and2

B are mutually exclusive events. Mutually exclusive events

are dependent as knowledge of event1B rules out event 2B taking place so that

P( 2B | 1B )=0 P( 2B )=1/6

Comment [S35]: P(E|F)=P(E)implies P(F|E)=P(F) and vice-versa


25/39



Page 25

(iii) If 1C =event sum of the outcomes of the 2 tosses is 6

Then it is not so obvious that event A and 1C are dependent. (NOT independent)

Lets try to reason it out.How can the sum of the 2 outcomes equaling 6 be formed ?The possibilities are{(1,5),(2,4),(3,3),(4,2),(5,1)}

Knowing event1C occurs rules out the possibility of obtaining a 6 in the first toss.

There are only 5 possibilities for the 1st toss {1,2,3,4,5} which are all equally likely.Hence, P(A| 1C )=1/5 whereas P(A)=1/6

(iv) If2C =event sum of the 2 tosses is 7, is this event independent from event A ?

You try it. Lets try to reason it out again.How can the sum of the 2 outcomes equaling 7 be formed ?The possibilities are{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)}

Knowing event 2C occurs hasnt restricted the possibilities for the outcome of 1st roll

Hence, P(A|2C )=1/6 whereas P(A)=1/6 so the events A and 2C are independent.

Sometimes it is not easy to deduce whether A and B are independent form intuition alone.Consider the following simple example.

Example 24Consider an experiment of tossing 2 fair coins.Let A be the event 1st toss is a headLet B be the event two outcomes are the sameDetermine whether A and B are independent.

Comment [S36]: Alternatively,

1

1

1

( | C )

( C )

(C )

({(4,2)})

({(1,5),(2, 4),(3,3),(4,2),(5,1)})

1/ 36 1

5 / 36 5

P A

P A

P

P

P

=

=

= =

Comment [S37]: Alternatively

2

2

2

( | C )

( C )

(C )

({(4,3)})

({(1,6),( 2,5),(3,4),(4,3),(5,2),(6,1)})

1/ 36 1

6 /36 6

P A

P A

P

P

P

=

=

= =

Comment [S38]: Notice this event isdifferent from the event one of the tosses

is H which is not independent of B


26/39



Page 26

Exercise 13

A fair die is thrown three times. Events A, Band Care defined as follows:A: the total score is an odd number;B: a six appears at the first throw;C: the total score is 13.

Calculate P(A) and P(A|B) and state whether A and B are independent.Calculate P(C) and P(C|B) and state whether B and C are independent.

Example 25A bag contains 5 white balls and 3 green balls. Two balls are to be drawn from the bag.Find the probability that the first ball is green and the second is white if(i) the balls are drawn one after another with replacement;(ii) the balls are drawn without replacement,.

Further Properties of independent events.Show that If A and B are independent then(i) A and B are independent(ii) A and B are independent(iii) A and B are independent.

Proof: P( ' P( ) P(A B A A B ) = )

( )

P( ) P( P(

P( ) 1 P(

P( )P( '

A A B

A B

A B

= ) )

= )

= )

Lets try to reason out the property above.If knowledge of B does not influence the probability of A then surely knowledge that Bdoes NOT occur, should not influence the probability of A either.


27/39



Page 27

Here are more worked examples to consolidate what weve learnt in this

Chapter.

5 Worked Examples

Lets consider the problem mentioned in Section 0 Introduction

Example 26The first problem de Mere considered asked the question, What is the probability ofgetting at least one six in four throws of a die?

Solution:P(at least one six in four throws of a die)= 1- P(Not getting a six in four throws of a die)= 1-P(Not getting a 6)4 By independenceof each throw

= 1-4

5

6

=671

1296>

1

2

Gambler has greater than half chance of winning.

We also asked, What is the probability of getting at least one double six in twenty fourthrows?

P(at least one double six in twenty four throws of a die)= 1- P(Not getting a double six in twenty-four throws of a die)

= 1-

2425

36

= 0.4913

Documents

Chapter S1 - Probability