Probability Reading Material for ISI ExamProbability Reading Material for ISI Exam 7 Ctanujit Classes Sample Random Sampling Consider a population with N, . We want to choose a random

Probability Reading Material for ISI Exam

1 Ctanujit Classes

PROBABILITY THEORY

Probability is a measure of uncertainty.

Example:

1. Drawing a number at random between 0 and 1 has probability 0. So for each number it has 0

probability and combinational form will give 0 probability as a whole.

2. P(Number drawn is ) between 0 and 1.

So, P(Number drawn is 1/2) .

Rules of Probability.

In an experiment let be the set of all outcomes. is called the Sample space.

A subset A of is called an event.

Event A occurs if any occurs.

To define probability measures we need to specify probabilities of a collection of events, .

@ is just a specified collection of subsets of the set .

Only sets in will be assigned probabilities.

Q. What properties must have?

Answer : be closed under finite unions and finite intersections of set in as well as under

complementation

An induction argument shows that if are sets in then so are

.

i.e., By, ……(i)

Also if …….(ii)

So from (i) & (ii)

A non−empty collection satisfying (i) & (ii) is called a field of subsets of or

an algebra of subsets of .


2 Ctanujit Classes

However an algebra is not sufficient to define a probability measure.

−field or −algebra: A non−empty collection of subsets of a set is called a −filed of

subsets of provided that the following twp properties hold.

(i) If A .

(ii) If is a sequence of sets in , then

both belong to .

Probability Measure:

A probability measure P on a −algebra or −field of subset of is a function such that,

i)

ii) P

iii) If is a sequence of mutually disjoint (exclusive) sets in , then

Let then

Power sets of

Properties of Probability Measures:

If , then

(i) For

(ii)

(iii)

(By D’morgan’s Law)


3 Ctanujit Classes

Can be extended to a larger number of events.

i.e.

For any event E.

For events A & B

……(1)

If . So, eq. (1) reduces to

Also,

Law of addition:

For any two events A & B.

Proof:

&

……(1)

Now,


4 Ctanujit Classes

Since

……(2)

Using (1) & (2), we have

And consequently

Ex:

Solution :

Homework:

Can be proved by induction on A.

Exercise : 200 people attended a dinner party.

of them ate potato salad (among other things)

of them ate chicken casserole.

of them had potato salad but not chicken casserole.


5 Ctanujit Classes

What proportions ate either potato salad or chicken casserole (or both)?

……(1)

Now,

So, using (i) & (ii)

Finite Probability : In a finite probability model, the sample space has a finite number of outcomes.

Say

Define simple events:

Ex: Toss 2 coins together


6 Ctanujit Classes

{exactly one head shows up}

Facts :

i) Any two different simple events are disjoints.

ii) Any event E is the union of simple events corresponding to outcomes contained in E.

Probability of any event in a finite probability model is the sum of probabilities of simple events whose

(Discrete) Uniform Probability Model

In a finite probability model if all simple events have equal probability then it is called a uniform

probability model.

If and the probability model is uniform then

In such a probability model, if an event E has k outcomes then

Exercise: Roll a balanced die. Probability of any number between 1 & 6 is 1/6.

,

P(die comes up with an even number)


7 Ctanujit Classes

Sample Random Sampling

Consider a population with N, .

We want to choose a random sample of n units from this population to study its characteristics. This is a

random experiments where the outcomes are samples of n units from the above population.

Q. A bowl has 100 marbles of which 50 are red, 30 blue and 20 green. Consider drawing 5 marbles at

random from this bowl. How are marbles drawn? How does the composition of the bowl change after

each draw?

Simple Random Sampling with replacement (SRSWR)

(i) Each marble in the bowl at the time of drawing has the same chance of being drawn at

each draw. After each draw, the color of the marble drawn is noted and the marble drawn

is returned to the bowl.

In this sampling composition does not change.

Simple Random Sampling without Replacement (SRSWOR)

Each marble in the bowl at the time of drawing has the chance of being drawn at each draw.

After each draw, the color of the marble drawn is noted. But the marble drawn is not returned to the

bowl. * i.e., contents of the bowl changes.

Ordered Sample:

If a set S has m distinct points and another set T has n distinct points, then the numbers of pairs

that can be formed is mn.

If we take then no. of k−tuples is .

Where (S) .

SRSWR:

Population has N distinct units. A sample of size n is drawn using SRSWR.

All the outcomes are equally likely. Each simple event has probability is

.

If an event E consist of k outcomes, then


8 Ctanujit Classes

Exercise: Roll a pair of balanced die once. What is the probability that the sum of the dots which show

up is 7?

Solution:

Possible favorable outcome

SRSWOR is a discrete uniform probability model also but the sample is an ordered samples

(n−tuples) where any unit can appear at most once.

Permutation: A permutation of n distinct objects is an arrangement of these in a particular order, say,

two objects.

A, B can be arranged as .

So, total i.e., 21.

Similarly, {A, B, C} can be arranged in 6 ways i.e, 31.

Number of distinguishable permutation of n object is

Consider a set S of N distinct elements. Select one object from it. Without replacing it to the set, draw

one from the remaining (N−1) objects. Continue until we have a sample of n objects. .

Outcome is an n−tuple

Total no. of such outcomes or samples is

Al outcomes are equally likely simple event has probability

Exercise: Consider n distinct boxes and n distinct balls. There are n! ways of throwing these balls into

boxes such that each box gets one ball. In such an experiment probability that ball numbered getting

placed into box numbered j is .

Arguments for this:


9 Ctanujit Classes

Remove the th ball & th box then we are left with (n−1) distinct boxes and (n−1) distinct ball.

Probability of the distinct event is

P(K specified balls are in K specified boxes)

Unordered Samples or Combination

A committee of 7 people needs to be formed from a house of 180 people. Committee formed by (151, 2,

7, 92, 57, 63, 10) is the same as that (2, 7, 10, 57, 63, 92, 151). Each permutation of the combination

leads to the same committee.

A poker hand of 5 cards is drawn from a standard deck of cards.

The hand arranged in any order means the same.

Exercise: Consider a population S of m distinct objects. SRSWOR of n objects

m(m−1)(m−2)…….(m−n+1) different samples (ordered n−tuples);

If we take one of the examples, it can be permuted in n! different ways, all leading to the same

combination.

There will be a total of m(m−1) ………(m−n+1) different combination or unordered examples.

Number of ways of choosing n places out of m is

In a population of M objects, m1 are of one type, m2 are type 2, ……., are of type k,

No. of ways of choosing n objects from this population such that n1 are of type 1, n2 are of type 2, ………

are of type k is

Exercise: If there are 8 men & 4 women among 12 volunteers, the no. of ways of choosing a team

consisting of 3 men and 2 women is


10 Ctanujit Classes

Q. What is the probability a team of 5 consist of 3 men & 2 women is

Conditional Probability

Toss a fair 10 times. Probability of any event is ratio of the number of outcomes favouring the

event to 210 (total no. of outcomes).

Q. What is the probability that the last toss came up head?

Q. What is the probability that the last toss came up heads given that there was at least one head in

the experiment?

Now we know that the sample space include (TT….TT), but all others are exactly likely.

P(last toss came up heads given that the experiment resulted in at least one head)

Definition:

Let A & B be two events such that P(A) > 0. Then the conditional probability of B given A is defined as

Ex: Toss a fair coin 10 times. What is the probability of getting exactly 5 heads?

Solution: Total outcome

Possible favorable outcome


11 Ctanujit Classes

Ex: A bowl has m marbles of which m1, are red and the rest blue, n marbles are drawn at random

without replacing. What is the probability that there will be k red marbles in the sample?

Solution:

Ex: For any day in September suppose the following probability model holds.

Bright Sunshine

Partly Sunny

Totally cloud.

Rain

,

(a) Given that it is cloudy, what is the probability that will rain?


12 Ctanujit Classes

(b) Given that it is partly sunny or has bright sunshine what is the probability that it will rain?

LAW OF MULTIPLICATION:

Generalizing to

Partition :

Events partition the sample space


13 Ctanujit Classes

Each outcome in belongs one and only one of the partitioning events.

For any event B,

Consider E such that P(E) > 0 and take as the partition of . Then

Bayes Theorem: If P(A) > 0 and P (B)> 0. Then

Proof:

Generalizes to

Ex: In a particular community of the people smoke. For a smoker there is a chance of

getting cancer, whereas for a non−smoker it is 20 . What is the chance that a cancer patient chosen

at random is a smoker?


14 Ctanujit Classes

Solution:

Ex: Toss a fair coin 3 times. Find

(a) P(2nd toss comes up head)

(b) P(2nd toss comes up head | 1st cost coming up head)

Solution:

(a)

(b)

A gives no information about the occurrence of B.

Definition: If

then we say that A & B are independent events. (probabilistic/statistical).

Definition: Two events A & B are independent if

Ex: Consider SRSWR & SRSWOR of sample size from a set .

Sample Events

{c, c} {(a, a)} {(a, b)} {(a, c)} {(b, a)} {(b, b)} {(b, c)} {(c, a)} {(c, b)}

Probability (WR)

1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9

Probability (WOR)

0 0 1/6 1/6 1/6 0 1/6 1/6 1/6

WR: {both draws produce ‘a’}


15 Ctanujit Classes

{2nd draw produce ‘c’}

A & B are independent events.

SRSWOR:

,

Now, A & B are not independent.

In fact

Note: If A & B are independent events with the probability they cannot be disjoint.

Independence .

Pairwise Independence of Mutual independence.

A, B & C are pair wise independent of A & B are independent and A & C are independent & B & C are

independent.

i.e.,

A, B & C are mutually independent if in addition to pair wise independence we have

.

Ex: .


16 Ctanujit Classes

A, B & C are pair wise independent.

n events are said to be mutually independent if for all subcollections

of size .

1. Toss a coin 3 times.

2. Choose a sample of size 2 using SRSWR for a population of size 100.

3. SRSWOR –no independence for draws P

4. Roll a die. If you hit a six, roll it again. If no six stop.

Ex: (1): Toss a fair coin 3 times. It is reasonable to assume that outcome of the first toss does not

influence that of 2nd toss, or 3rd, outcome of 2nd does not influence that of 3rd, so mutual independence

of components of the experiment may be assured.

for each toss

Ex (2): Draw a sample of size 2 from population of 100 using SRSWR.

In Steps :


17 Ctanujit Classes

Ex(3): Same with SRSWOR

Elements of Combinatorial Analysis

RULE –I : If there are two groups G1 & G2; consisting of n elements and

consisting of m elements then the no. of pairs formed by taking one

element .

If there are k groups , such that

Then the number ordered k−tuples formed by taking one element from each group is

Example : ‘Placing balls into the cells’ amounts to choose one cell for each ball. Let there are r balls and

n cells. For the 1st ball, we can choose any one of the n cells. Similarly, for each of the balls, we have n

choices, assuming the capacity of each cell is infinite or we can place more than one ball in each cell.

Hence the r balls can be placed in the n cells in ways.

Applications:

1. A die is rolled r times. Find the probability that –

i) No ace turns up. [ace −1]

ii) No ace turns up.

Solution:

i) The experiment of throwing a die r times has possible outcomes.


18 Ctanujit Classes

Assume that all possible cases are equally likely. The no. of cases favorable to the event (A), ‘no ace

turns up’ is .

By Classical Definition,

ii) P[ an ace turns up]

Remark : The all possible outcomes of ‘r’ throw of a die correspond to the placing r balls into cells.

RULE –II:

Ordered Samples: Consider a population of n elements any order arrangement

of r elements is called an ordered sample of size r, drawn from the population. Two

procedure are possible –

i) Sampling with replacement: Here an element is selected from the population and the

selected element is returned to the population before the next selection is made. Each

selection is made from the entire population, so that the same element can be drawn more

than ones.

ii) Sampling without replacement : Here an element once chosen is removed from the

population, so that the sample becomes an arrangement without repetition.

For a population with n elements and a prescribed sample size r, there are different ordered

samples with replacement and n(n−1)…..(n−r+1) different ordered samples

without replacement.

Remark:

1. is defined if and r is a non−negative integers. But

is defined if and r is non−negative integer. In the same

way if then

Example : 1) A random sample of size ‘r’ with replacement is taken from a population of n elements.

Find the probability that in the sample no element appear twice.

Solution: There are sample in all. As the samples are drawn randomly, all samples are equally likely.

The no. of the samples in which in which no element appears twice is the no. of samples drawn without

replacement.


19 Ctanujit Classes

Favorable sample is

Hence, the probability is

Example: 2) If n balls are randomly placed into n cells, what is the probability that each cell will be

occupied.

Solution:

SOLVED EXAMPLES:

1. Find the probability that among five randomly selected digits, all digits are different.

Ans:

2. In a city seven accidents occur each week in a particular week there occurs one accidents per

day. Is it surprising?

Ans :

3. An elevator (lift) stands with 7 passengers and stops at 10th floor. What is the probability that

no two passengers leave at the same floor?

Solution:


20 Ctanujit Classes

4. What is the probability that r individuals have different birthdays? Also show that the

probability is approximately equal to . How many people are required to make

the prob. of distinct birthdays less than ½ ?

Solution:

More than 23 people are required.

5. Six dice are thrown. What’s the prob. that every possible number will appear.

Hints:

6. There are four children in a family. Find the prob. that

(a) At least two of them have the same birthday?

(b) Only the oldest and the youngest have the same birthday?


21 Ctanujit Classes

Hints: (a)

(b)

7. The number 1, 2,….n are arranging in a random order. Find the probability that digits (a) 1, 2 ,

(b) 1, 2, 3 appears as neighbours in the order named.

Hints: consider (1, 2) as a single digit then there are (n−1) entities which can be arranged in (n−1)! ways.

(a) Required prob. is

.

(b) Required prob. is

8. (i) In sampling with replacement find the prob. that a fixed element be included at least once.

(ii) In sampling without replacement find the prob. that a fixed element of a population of n

elements to be included in a random sample of size r.

Hints:

(i) [ the fixed element is not included in the sample WOR]

(ii) [ a fixed element is not included in the sample WR]

9. There is 3 volume dictionary among 30 books is arranged in a shelf in random way. Find the

prob. of 3 volume standing in an increasing order from left to right? (The vols. are not

necessary side by side).

Solution: The order of the 3 vols. doesn’t depend on the arrangement of the remaining books. Here 3

vols. can be arranged in 3! ways of which only one case is favorable. Hence prob. is 1/3!.

10. Two fair dice are thrown 10 times. Find the prob. that the first 3 throws result in a sum of 7

and the last 7 throws in a sum of 8.


22 Ctanujit Classes

Solution: , be the sample space of the kth throw of a pair of dice,

the sample space of the experiment is

Let, , the event of getting a sum of 7 in a throw of a pair of dice.

And , the event of getting a sum of 8 in a throw of a pair of dice.

Our event is

Favorable cases are

11.

(i) If n men, among whom A and B, stand in a row. What’s the prob. that there will be exactly r

men between A and B?

(ii) If they stand in a ring instead of in a row, show that the prob. is independent of ‘r’.

[In the circular arrangement, consider only that they are leading from A to B in the +ve direction.]

Solution:

(i) n persons can be arranged among themselves in n! ways. Since, the persons are randomly,

all possible cases are equally likely. For the favorable cases if A occupies a position to the left

of B, then A may choose any of the positions:

1st, 2nd,….. (n−r−1)th from the left, with r persons between A and B. The remaining (n−2) persons can

stand in (n−2) places in (n−2)! Ways. Similar thing for B on the left of A.

Hence, no. of favorable cases,

Required probability

(ii) If they form a ring, then the no. of possible arrangement is (n−1)! which is obtained by

keeping the place for any person fixed and arranging the remaining (n−1) persons.


23 Ctanujit Classes

For the favorable cases, we fixed the places for A and B, with r individuals between them and then

remaining (n−2) persons can be arranged in (n−2)! ways.

RULE−III:

Subpopulations and Groups: Consider a subpopulation of size ‘r’ from a given population of size ‘n’, let

the no. of the groups of size r be x.

Now the r elements in a group can be arranged in r! ways. Hence x . r! ordered samples of size r.

Application :

1. Each of the 50 states has two senator. Find the prop. of the event that in a committee of 50

senators chosen randomly –

(a) A given state is represented.

(b) All states are represented.

Solution: We can choose a group of 50 senators in ways & since 50 senators are chosen randomly

50 all possible outcomes are equally likely.

(a) There are 100 senators and 98 not from the given state.

Required probability [the given state is not represented] C

(b) All states will be represented if one senators from each state is selected. A committee of 50 with

one senator from 50 states can be selected in

ways.

2. If n balls are placed at random in n cells, find the probability that exactly one cell remains

empty.


24 Ctanujit Classes

Solution:

Since k balls can be chosen in ways which are to be placed in the specified cells and the remaining

(r−k) balls can be placed in the remaining (n−1) cells in ways.

3. If n balls are placed at a random order in n cells, find the prob. that exactly one cell remains

empty.

Solution:

For the favorable cases, the empty cell can be chosen in n ways and the two balls to be kept in the same

cell can be chosen in ways.

Consider the two balls as a single ball or entity, then (n−1) entities can be arranged in (n−1) cells in

(n−1)! ways.

So, the required prob.

4. A closent contains n pairs of shoes. If 2r shoes chosen at random (2r < n). What is the prob.

that there will be:

(a) No complete pair

(b) Exactly one complete pair

(c) Exactly two complete pair among them.

Solution: (a)

(b)


25 Ctanujit Classes

(c)

5. A car is parked among N cars in a row, not at either end. On the return the car owner finds

that exactly r of the N places are still occupied. What’s the prob. that both neighbouring

places are empty?

Solution:

RULE –IV:

The no. of ways in which a population of n elements can be divided into K−ordered parts of which 1st

contains , 2nd contains r2 elements and so on is

Application:

1. In a bridge table, calculate the prob. that

(a) Each of the 4 players has an ace

(b) One of the player receives all 13 spades.

Solution:

(a) In a bridge table 52 cards are partitioned into four equal groups and the no. of different hands is

For the favorable cases, 4 aces can be arranged in 4! ways and each arrangement represents one

possibility of given one ace to each player and the remaining 48 cards can be distributed equally among

the 4 players in


26 Ctanujit Classes

(b)

2. In a bridge hand of cards consists of 13 cards drawn at random WOR from a deck of 52 cards.

Find the prob. that a hand of cards will contain

(a) clubs, spades, diamonds

(b) aces

(c) aces and kings.

Solution: (a)

(b)

(c)

3. 4 cards are drawn at random from a full deck of 52 cards. What’s the prob. that

(i) They are of different denominations?

(ii) They are of different suits?

(iii) Both?

Solution:

(i) In a deck of cards there are 13 denominations and 4 suits.


27 Ctanujit Classes

For favorable cases select a group of 4 denominations from 13 and then choose one card from each of

the 4 denomination.

So , no. of favorable cases

.

(ii)

(iii) For favorable cases, selecting 4 denominations from 13 and then taking one card from the

1st denomination in 4 ways from the 4 suits. Then taking 2nd from the 2nd denomination in 3

ways & so on.

4. From a deck of 52 cards are drawn successively until an ace appears. What is the prob. that

the 1st ace will appear

(a) At the nth draw,

(b) After the nth draw.

Solution:

(a) For the favourable cases, at the nth draw an ace can occur in 4 ways and the first (n−1) cards are

to be taken from 48 non−ace cards which can be done in ways.

(b) For the favorable cases, 1st n cards contain no ace.

5. (Spread of Rumours) In a town of (n+1) inhabitants, a person tells a rumour to a second

person, who in turn, repeats it to a third person, etc. At each step the receipt of the rumour is

chosen at random from n people available.

(i) Find the prob. that the rumour will be told r times without


28 Ctanujit Classes

(a) Returning to the originator.

(b) Being repeated to any person.

(ii) Do the same problem when at each step the rumour is told by one person to a gathering

of N randomly chosen individuals.

Solution:

(i) Since any person can tell the rumour to any one of the n available persons in n ways, total

possible cases .

(a) The originator can tell the rumour to anyone of the remaining n persons in n ways & each of the

(r−1) receipts of the rumour can tell to anyone of the remaining (n−1) persons without returning

to the originator in (n−1) ways.

(b)

(ii)

(a)

(b)


29 Ctanujit Classes

6. 5 cards are taken at random from a full deck. Find the probability that

(a) They are different denominations?

(b) 2 are of same denominations?

(c) One pair if of one denomination & other pair of a different denomination and one odd?

(d) There are of one denomination & two scattered?

(e) 2 are of one denomination and 3 of another?

(f) 4 are of one denomination and 1 of another?

Solution : (a)

(b)

(c)

(d)

(e)

(f)


30 Ctanujit Classes

RULE−V:

Occupancy Problem: In many situations it is necessary to treat the balls indistinguishable, e.g., in

statistical studies of the distribution of accidents among week days, here one is interested only

in the number of occurrences and not in the individual involved.

Such an example is completely described by its occupancy numbers ; where, denotes the

number of balls in the kth cell.

Here we are interested in number of possible distribution, i.e., the number of different n−tuples

such that .

Theorem 1: The number of different distributions of ‘r’ indistinguishable balls in n cells, i.e., the

number of different solution of the above fact is

Theorem 2: The number of different distribution of ‘r’ indistinguishable balls in the n cells in

which no cell remains empty is

Ex: r distinguishable balls are distributed into n cells and all possible distributions are equally likely.

Find the prob. that exactly m cells remain empty.

Solution: The m cells which are to be kept empty can be chosen from n cells in ways and r

indistinguishable balls can be distributed in the remaining (n−m) cells so that no cell remain empty is in

No. of favorable cases

Application:

1. Show that r indistinguishable balls can be distributed in n cells i.e., the no. of different

solution such that is

, where .


31 Ctanujit Classes

Solution : Denoting the choices of i.e., 0, 1, ….., r in the indices, we get the factors

.

The no. of different solutions of

The coefficient of in


The coefficient of in the expression


2. Show that the no. of different distributions of r indistinguishable balls in n cells where no cell

remains empty is

.

Hints:

The coefficients of in


The coefficient of


32 Ctanujit Classes

Random Variables:

Ex: Suppose a machine produces bolts, of which are defective. Find the probability that a box of 20

bolts contains atmost one defective bolt.

Discrete Random Variables:

e.g. : Toss a fair coin 4 times – discrete uniform model

equally likely outcomes.

X of heads

Possible values are 0, 1, 2, 3, 4

if x is not 0, 1, 2, 3, 4.

Definition: The real−valued function f or p defined on R by is called the

probability mass function or probability discrete density function or probability density function of X.

e.g.:


33 Ctanujit Classes

Ex: random integer between 0 & 99 –discrete uniform

Ex: of tosses until a head shows up (in an experiment of tossing a fair coin).

Properties of P.M.F

i)

ii) support of X is a finite or a countable finite set.

iii)

Examples of Standard discrete probability distribution

1. Bernoulli random Variable & Bernoulli distribution :

Let A be an event with

X is the indicator function of A.


34 Ctanujit Classes

A occurs if 1

Ex: Toss a coin once. Let if head comes up, 0 otherwise. Let the coin have probability p of coming

up heads on any toss. Then X is Bernoulli (p) or Bernoulli (p).

For a fair coin,

.

Ex: Roll a balanced die once. Let of six dots show up, 0 otherwise.

Bernoulli (1/6).

Let

Bernoulli (1/2).

Binomial Distribution:

Toss a fair coin n times. All outcomes are equally likely. Let of heads in these n tosses.

X can take values 0, 1, 2, 3,…, n.


35 Ctanujit Classes

Ex 2: Roll a balanced die n times.

of 6’s. What is the probability f(x) ?

of accidents

of radioactive particles emitted during an interval.

Poisson distribution

Intensity rate :

Poisson ( ) if

This f(x) is probability mass function because

i)

ii)

It is convenient to define the function


36 Ctanujit Classes

Is called (c.d.f) cumulative distribution function or distribution function (d. f)

For any f,

For any .

F is non−decreasing since

Ex: Toss a fair coin 4 times.

of heads

In this example, F is continuous everywhere except at and at these points just size is


37 Ctanujit Classes

In this example, if F is given,

Jointly Distributed Random Variables:

Simultaneously distributed random variables.

Ex: Roll a pair of balanced dice.

of sixes

Sum of the numbers that show up.

A random vector is a set of simultaneously defined random variables.

is called p.m.f or p.d.f.

Marginal Probability distribution

Ex: Roll a pair of balanced dice

of dots a first dice

of dots on 2nd dice.

Sum of the dots


38 Ctanujit Classes

e.g.

& show

Roll a pair of balanced dice

dots on first die

dots on second die


39 Ctanujit Classes

Joint p.m.f.

Marginal p.m.f of x,

sum of the dots on 1st &2nd

Given the joint p.m.f. of (X, Y), the marginal p.m.f. of x & y are uniquely determined. However, marginal

p.m.fs do not determine the joint p.m.f. in general.


40 Ctanujit Classes

Independent Random Variables:

Consider a bowl with 100 marbles of which 20 are red. Go on drawing one at a time using SRSWR.

Define

are identically distributed.

Since this is simple Random sampling with replacement, outcome of the ith draw can have no influence

on the outcome of any other draw. .

Therefore,

The events are independent if that is

Ex. : Roll a balanced die twice.

of dots in the first roll

of dots in the second roll.

can be modeled to be independent.

Def: Let be r discrete random variables having p.m.f. respectively.

Then are said to be mutually independent if the joint p.m.f.

satisfies

Suppose X & Y are independent with joint p.m.f. & marginal p.m.f. . Then


41 Ctanujit Classes

Generalizing if are mutually independent random variables, then

2 = 1 1. 2 2 .

Reason for this:

Ex: (1)

Are X & Y independent?

Ex: (2)

are integers

Are X & Y independent?

Examples: Consider a sequence of independent and identically distributed Bernoulli (p) random

variables . Then for any binary sequence of length n.


42 Ctanujit Classes

number of success in a (S, F) sequence where

For 0, 1, …., n

If

.

It is possible to construct an infinite sequence of i.i.d. Bernoulli (p) random variables:

,… for any positive integer n, are mutually dependent.

Consider such a sequence if i.i.d. Bernoulli (p) random variable.


43 Ctanujit Classes

Let of 0’s before the first 1 in the sequence.

geometric (p)

If k is a non negative integer,

Suppose the lifetime of a device (in whole units) is modeled using the geometric (p) distribution.

of units until the device fails.

What is implication of such a model?

For +ve integers K & , consider

the device will not fail until K+ units given that it has not failed by time K)


44 Ctanujit Classes

Geometric (P) if 0, 1, 2,…

of failures before the 1st success.

(success)

Negative Binomial: Consider again an infinite sequence of i.i.d. Bernoulli (P) :

Let be any positive integer.

number of failures before there are r successes

of 0’s in the sequence until there are exactly r 1‘s for the first time.

Where is the number of 0’s after the th 1 and before the ith 1.

are iid geometric (p).

K 0’s are distributed in k+r−1 places in the binary sequence).

Negative Binomial geometric (p)

Hyper geometric distribution

Recall the example where we consider a bowl with m marbles of which mp are red in colour and m(1−p)

blue. n marbles are drawn at random without replacement.

of red marbles in the sample.


45 Ctanujit Classes

Bernoulli (p), are Bernoulli (p)

But they are not independent.

What will happen if m is very large compared to n?

Then removing any fixed number of marbles upto n will not change the constitution of the bowl at any

draw significantly.

are approximately iid Bernoulli (P).

Hyper geometric distribution can be approximated by the Binomial distribution or SRSWOR SRSWR.

Multinomial Distribution

Ex Roll a balanced die 20 times and

Let of times dots showing

For


46 Ctanujit Classes

Here n 20

In general, let an experiment have r possible outcomes.

Let ,

Repeat this experiment n times and define

of times jth outcome occurs,

( of sequences such that there are 1’s, 2’s ,….., r’s)

Define


47 Ctanujit Classes

Then are iid Bernoulli (p1) since

Marginals of multinomial are Binomial.

Functions of Random Variable (discrete)

Ex: Bernoulli (p).

is also a Bernoulli i.e., Bernoulli (p).

Ex: Binomial (n, p)

Let X have p.m.f. and let for some function g.

Then

: ( )=


48 Ctanujit Classes

Ex:

Sum of independent discrete random variables

Consider (X, Y) with joint p.m.f. .

Let X take values Y take values

Let , then

Suppose X & Y are independent

Then


49 Ctanujit Classes

Ex: Suppose Binomial (n1, p) , Binomial (n2, p) & X & Y are independent.

Show Binomial .

For

If sum is over all x such that (*) holds.

So,

Binomial (n1, p) of successes in trials where

Binomial (n2, p) of success in the next n2 trials

of successes in trials. .


50 Ctanujit Classes

Ex: . X & Y are independent. Show .

Expectation of Discrete Random Variables

Ex: In a population there are 3 grades of workers, A , B & C who collect fruits. The properties are 20 ,

50 . A grade workers collect 100 kg of fruits per day, B grade 60 kg & C grade 40 kg. What is the

average collection of a randomly chosen person (per day)?

Solution:

100 workers

Collects

So average

kg

Collection of a Randomly chosen worker

X takes 3 values 100, 60 & 40.

Expectation or expected value or Mean of a discrete random variable.

In general, if X is a discrete r.v with p.m.f. , then the expected value of X is defined as


51 Ctanujit Classes

e.g. Suppose X is uniformly over the integers 0, 1, 2, ….., 100 i.e.

In general, if X is uniform over

Ex: X Bernoulli (p). Then

Ex: Binomial (n, p). Then

Ex: X Geometric (p). Then


52 Ctanujit Classes

Ex: X Poisson

Then

Show

Ex:

If


53 Ctanujit Classes

Result:

If X is a discrete r.v. with p.m.f. and g is a real−valued function such that

Then for

If summation goes to we generally don’t define E(X).

Result: If is a discrete random vector having p.m.f.

Theorem:

Let X & Y be two random variables with joint p.m.f. and suppose E(X) and E(Y) exist.

i) If for some C, ii) For any c constant, exists and

iii)


54 Ctanujit Classes

iv) If P(X , then , with equality if .

Proof: ,

v) .

, then

Suppose X takes values,

Similarly


55 Ctanujit Classes

Ex: X Bernoulli (p), .

Result: If X & Y are independent with p.m.f. and they have finite expectations, then

Proof :

But converse may not be true (in general).

If , then X & Y need not be linearly independent.

E(X)

with probability 1

mean of X

Ex: X Binomial (n, p)


56 Ctanujit Classes

i.i.d Bernoulli (p)

.

Ex: X NB (r, p)

Ex: A bowl has m marbles out of which mp are red, rest blue.

a) Draw n marbles at random using SRSWR from this

of red marbles in the sample

b) Draw n marbles at random using SRSWOR.

X of red marbles in the sample,

X HG (m, p ; n)

Bernoulli (p)

Poisson approximation for the Binomial Bin (n, p) n is large, p small, no is moderate.

Consider the number of radioactive properties emitted by a source during a unit time interval. Suppose the average rate is (we would model the count as Poisson ( )).

Total of particles emitted sum of counts in the subintervals

Sum of n Bernoulli trials

Binomial

is large enough so that there will be at most one particle emitted is any subinterval.


57 Ctanujit Classes

What happens as ?

Probability that a Poisson takes value k

Ex: X Binomial (n, p),

MOMENTS

Let X be a discrete random variables with p.m.f .

If X has finite expectation we denote it by .

For any integer if has finite expectations, the rth moment (raw moment of X is defined as


58 Ctanujit Classes

If exists, rth moment of is called the rth central moment of X.

Result: If rth moment exists, rth central moment exists, and vice versa.

Result: If rth moment exists, then all moments of order k, exist.

Result: If X & Y have moments of order r, so does X+Y.

Def: If X has finite second moment, then its second central moment is called its variance.

Def: Standard Deviation of X is defined as

Result: for some C.

If , then

Then,

If , then


59 Ctanujit Classes

Suppose X has mean . Thus

Is called the standardized form of X.

Variation of the sum. Let X & Y be two random variables with finite second moment. Then X+Y has 2nd moment and

Covariance between X & Y :


60 Ctanujit Classes

Suppose,

Then .

As X increases does Y typically increase?

Is there a positive associate between X & Y?

If X & Y are independent,

If are mutually independent.

iid or a random sample from a population with mean .


61 Ctanujit Classes

Ex: Y Bernoulli (p). Then

iid Bernoulli (p).

Each has mean p & variance p(1−p)


62 Ctanujit Classes

Ex: Show that if X Poisson .

−depends on the scaling or units used by X & Y.

i.e.

is a standardized measure derived from Cov (X, Y) which does not depend on scaling.

is called the correlation coefficient between X & Y.

Theorem :

for some constant a.

Proof: Hint

Cauchy−Schwarz inequality . If X & Y have second moments,

Then

Apply this to r.v. :


63 Ctanujit Classes

Q. If X & Y have finite second moments then if in addition X & Y are independent.

is a measure of dependence between X & Y.

However, X & Y need not to be independent if .

, for some constant a.

So measures linear association between X & Y.

CONTINUOUS RANDOM VARIABLE

Many random variables are best treated as continuous, in the sense, any value in an interval of values is a probability.

lifetime of a light bulb.

Probability of T being any fixed value is O, but the probability of T being in a subinterval is positive.

waiting time at a service counter.

A random number between 0 & 1

Point where the arrow in a spinning wheel stops.

Arrival time of radioactive particle

For a continuous Random Variable X

for any x

How do we describe its probability distribution?

We do so by describing probability such as

Def : A random variable X on a probability space is a real−valued function

Such that for all

With this distribution we can proceed by noting that the cdf of X,

is available

Then, for any


64 Ctanujit Classes

Ex: For of particles

Emitted by a radioactive substance in the time interval [0, f]. Suppose N

Waiting time for the first particle to arrive

No rounding off or approximation is involved.

Ex: Random number between 0 & 1.

Poisson (2)

How does one get the probability density function from its cdf?

Continuous Probability Distributions

If X is any random variable and F is its c.d.f. then

(i)


65 Ctanujit Classes

(ii) F is non−decreasing (iii) F(− , F (iv) F(x+) light continuity

Since

F has a jump discontinuity at x; otherwise if , F is continuous at x.

For a discrete random variable, its c.d.f. F has a countable no. of jump discontinuities

A random variable is continuous (i.e., ) if, its cdf is continuous at every x, i.e., from (*),

Definition:

The probability density function (when it exists) of a continuous random variable x, with cdf F is a non−negative function, F such that

Since,

(at all the points x where f is continuous)

Now, F is the area under the curve of . Since, area over any point is 0; f can be arbitrarily defined at many points. Also, F must be differentiable almost everywhere.

If X is continuous with p.d.f. f,


66 Ctanujit Classes

Ex:

random variable between 0 & 1

i.e., X is uniform over [0, 1]

Ex:

Thus

Exponential

is the p.d.f of a continuous random variable X.

What does this mean?

Suppose

For any


67 Ctanujit Classes

lifetime of a light bulb

(Memory less property)

is called the parameter of the distribution

e.g. X

For

Put


68 Ctanujit Classes

Now,

But

i)

So,

ii)

iii)


69 Ctanujit Classes

Properties of Gamma Distribution

i) Gamma

ii) For

The gamma p.d.f. is unbounded near O.

Except for certain values of ,

is not available in closed form.

If is an integer, then

Repeating the lines


70 Ctanujit Classes

If are iid Exp then,

Ex: X if

Where are parameters of distribution

i)

ii)

iii)

Unbounded near O

Ex: X

is called the standard normal distribution

Q (z)

Density of N (0, 1) : Q (z)

cdf of N(0, 1)


71 Ctanujit Classes


72 Ctanujit Classes

Functions of Random Variables

Ex: Suppose X

What is the probability distribution of Y?

Solution:

For

Find the probability distribution of

Ex: Suppose

Then

For y > 0


73 Ctanujit Classes

Chi –square with k degrees of freedom

Change of Variable Formula

Find the p.d.f. of where the p.d.f. of X is given and g is I−1 real value function.

Theorem: Let g be a differentiable strictly monotone (strictly increase or strictly decreasing) function on an interval I. Let g(I) denote the range of g and let denote the inverse of g. Let X be a continuous random variable having p.d.f .

Then has p.d.f given by

Proof: Let denote the cdf of X & Y, respectively consider the case when g is strictly increasing. Then is also strictly increasing.


74 Ctanujit Classes

If &

Since

For

Show this result when g is decreasing.

Ex: X

Let


75 Ctanujit Classes

For a < 0

Result: Let X be a continuous random variable with pdf . For constants a & b, a , let

Then

Ex: Let .


76 Ctanujit Classes

What is the median income of an Indian family?

In [2011−12, 90500 per annum]

income of an Indian Family

continuous strictly increasing in an interval I

Given 0>p<1 there exists

Such that

Expectation of a Continuous Random Variable

If Y is a discrete with p.m.f f(y) and

If Y is continuous with p.d.f f then


77 Ctanujit Classes

i.e., replace sum by integral

Def: Let X be a continuous r.v. with p.d.f .

Then we say that X has finite expectation (expected value) and define its expectation by

Ex:

Ex: If U

Ex: If X then


78 Ctanujit Classes

[b−a length of (a, b)]

OR

Ex: X

By his result

If Y is a continuous r.v. with p.d.f. such that and if Y has finite expectation, then .

Let


79 Ctanujit Classes

So (1) reduces to using (2)

Ex: X

Show

More generally, if then

For a continuous random variable X with p.d.f. , and mean if X has finite 2nd moment then

Then the variance of X is given by


80 Ctanujit Classes

Ex: U

So

Ex: If X

Then

Then

Ex: X

Then ,

Let


81 Ctanujit Classes

Ex: X

Continuous bivariate distribution

Let X & Y be jointly distributed with cdf F(x, y)

Marginal cdfs of X & Y are given by

Joint p.d.f. of (X, Y) is given by f(x, y)

Where


82 Ctanujit Classes

Similarly

Ex: (X, Y) coordinates of a randomly chosen points inside the unit disk

Find

Sol. Since

Documents

Probability Reading Material for ISI ExamProbability Reading Material for ISI Exam 7 Ctanujit Classes Sample Random Sampling Consider a population with N, . We want to choose a random