01 Principles of Probablity 1

8/3/2019 01 Principles of Probablity 1

1/39

Experiments, Sample Spaces & Events

[Reading: SG ; Sections 1.1 - 1.2]

(1) Experiments

An experiment (with random outcomes) consists of:

a repeatable experimental procedure and

a specified set of outcomes S (also called sample

points or (singleton events or observations) such that

the outcomes vary unpredictably between successive

trials (i.e. repetitions of the experiment)


2/39


3/39

Example E1: Single Coin Toss

Experimental Procedure of E1 = Toss a coin once

Observation (i.e. Sample point) = H (Heads) or T (Tails)

Example E2: Double Coin Toss

Experimental Procedure E2 = toss two coins (i.e. 2 E1simultaneously).

Observation (i.e. Sample point) = the pair of faces

(each H or T) showing as a result of the experiment

NB: Pair of faces of each sample (point) is a two component vector.


4/39

Experiment = MissionWhat is the probability of the outcome: Safe completion

of mission: 19/20? or 57/60?Or of the outcome: Catastrophe 1/40? or 1/400?Are the experiments in this case genuinely repeatable?


5/39

Path Prediction for Hurricane Kenneth 25/09/2005

Experiment = Repeated viewing of hurrican trajectories

starting in same region.Possible Outcomes = Points in Cone Reached at Specified

Time/Date


6/39

Experiment = Repeated plays of roulette.Possible Outcomes = Slot in which ball stops.

Fraction of times ball falls in any slot is engineered to be

the same. But rewards for correct play are engineered

so that on average (i.e in the long run) player loses.


7/39

(2) Sample Space [SG p.3]

The sample space S of an experiment

= the set ofoutcomes of the experiment (or the

set of sample points or set of singleton events)


Experiment E2 = toss two coins (i.e. E1 twice).

Observation = H or T face of each coin

Sample Space S = { (H,H), (H,T), (T,H), (T,T) }


8/39

Example Esum: Double Toss Summing Face Values

Experiment Esum = toss two coins.

Observation = count number of HsSample Space S = { 0, 1, 2 }

Common Sense Probabilities:

P(Head Count = 0) =1

4

P(Head Count = 1) =1

2

P(Head Count = 2) = 14


9/39

Example E100sum: 100 Repetitions of Esum

Observation = Sequence of 100 Repetitions of Esum

= a sequence of 0s, 1s, 2s of length 100

Sample Space S = {{0,1,2}, {0,1,2}, .....,....,{ 0,1,2 } } (100 times)

{ 1001 {0,1,2} }


10/39

Example Etranslife: Lifetime of a Transistor, or a

Beer Bubble

Experiment Etranslife = measure life time of a transistor,

or a beer bubble.

Observation = time until failure (hours), or

bursting (in minutes)

Sample Space S = { t ; 0 t < }

Notice! The sample space is a continuous set, not a finitecollection of elements.


11/39


12/39

Example E: Infinite Sequence of Coin Tosses

Experiment E = repeat E1 indefinitely

Obs. = Seq. of Hs, Ts along an infinite run of tosses

repeat single coin toss without termination

Sample Space S = {H, T} {H, T} ....

Everyday Example of E: Sequence of Digital Bits

in any Communication or Computation System

Notice! Only one set of probabilitiesconsidered so far;

mainly experiments, outcomes and sample spaces.


13/39

(3) Events [SG pp.4]

The set of events E(S) is the collection of all subsets of

S.

Here we assume |S| < , i.e. S is a finite sample space,but shall see that subject to a standard assumption the

definition holds for infinite sets.Example E1: Single Coin Toss

Sample Space S = {H,T} ; Event Set E1(S) = { ,H,T, { H,T } } { no observation , H obsd. ,T obsd.,

H or T obsd. }

Example of a Single (Specified) Event in E1(S) :

{H} = { H is observed at coin toss }


14/39


Sample Space S = { (H,H), (H,T), (T,H), (T,T) }

Example of a Single Event in E2(S) ( recall, E2(S) = all

non-empty subsets of S together with ) :

HF { (H,H), (H,T) } { H appears on first coin }

Common Sense Probabilities (Fair Coin):

P((H, H)) = 14

; P(HF) = 12

; P(At least oneH occurs) = 34


15/39

Combinations of Events

Let E S, F S. (Note: same as E,FE(S).) ThenE F S, i.e. E F E(S)

E F S is interpreted as the event: the outcome ofthe experiment is in E or is in F

E F S is interpreted as the event the outcome of theexperiment is both in E and in F

Ec S is interpreted as the event the outcome of the

experiment is not in E

See that the combination (union, intersection,

complement) of events are events.


16/39


{(H,H) (H,T) } = { (H, H T) }

{ (H first, then any face ) }

{ (H,H) (H,T) } = { (H, H T)} =

{ (H, ) } { the null event }

since (H, ) does not lie in the collection of all possible

sample points (H,H), (H,T), (T,H), (T,T).


17/39

Example: Sets from Combinations of Simpler Sets

Five On-Off Units: Denote the Idle, resp. Busy, states

of the components respectively by the symbols 0, and 1.

S: set of all possible states: |S| = 25 = 32

Set A4 S = {4 or more components Busy }

{ 01111 10111 11011 11101 11110 11111 }

The event A4E(S) is composed of sample points

themselves formed from the combination of simple states.

Let the event I1E(S) consist of those sample points forwhich exactly one component is Idle, i.e. exactly four are

Busy; then I1 A4. Here P(I1) =5

32while P(A4) =

6

32


18/39


19/39


20/39

Example INTERNET ADDRESSES

The following assertions may be verified using the basic

definitions of the subset relation and the complementaryproperty.

Consider sets of addresses with usernames:

jack, jill, harry, helen

Suppose they can all use the machines:ampere, faraday, gauss, hamilton

Assume everyone can give the suffices:.ca or .edu


21/39

List of possible e-mail addresses:

S = { [email protected], ...,...,[email protected] }

Then |S| = 32 and |E(S)| = 232, since any particularsubset T is specified by a list of 32 Yes or No

symbols corresponding to the inclusion or exclusion of thecorresponding address from T.


22/39

With |S| = 32 and |E(S)| = 232 consider searchingat random for an address which one knows lies in the set

Addseek E(S), where Addseek is defined to be the set

{names begins with j } { Irish machine }

{ Canadian suffix}

= { [email protected], [email protected]}

So our chance of finding an address fitting the

description of Addseek if we pick out single addresses

at random (rather than use a systematic search) is 2/32.

If we wanted the set Addseek and took sets atrandom our chance of getting Addseek is 1/232.


23/39

Random Processes [SG pp. 511 - 512]

A random process is an experiment where the outcome

of each trial is a time indexed set of observations, i.e. a

sequence of observations, (termed a sample path).

Example E

: Infinite Sequence of Coin TossesExperiment E = repeat E1 indefinitely

Obs. = Seq. of Hs, Ts along an infinite run of tosses

repeat single coin toss without termination


24/39

Example: Economic Time Series; Interest Rates

Experimental Observation = One (2-component)

sample path. Note the explicit time indexing on the

horizontal axis. Problem: Introduction of a new currencyis a difficult experiment to repeat.


25/39

Path Prediction for Hurricane Kenneth 25/09/2005

Random Process Outcomes = Alternative Paths over 3

and 5 Days


26/39

A Collection of Random Trajectories of Hurricanes in 2004


27/39

Non-Example?: Historical Temperature Records


28/39

Events for Random Processess

In the case of a random process the set of events is thecollection of all possible sets of sample paths.

Hurricane Example: A specific event:

{The set of all hurricane sample paths (i.e. trajectories)which begin in the specified initial coordinate square on

Day 1 and lie in the coordinate square containing the Big

Island (Hawaii) on Day 6.}

Historical Temperature Record Example: A specific event:

{The set of all (green path) records all of whose values

lie above -0.3 after 1850.}


29/39

Problem: There is only one historical sample path in the

second case above! Question: So how do we interpret

such terms as fraction of sample paths on which some

specified observation happens for such unrepeatableexperiments?

Answer: With great difficulty for clearly unrepeatable

experiments; for example, history itself!

One plausible answer to the Question is given in

situations such as gambling or insurance, where one

has partial information, but no extensive experimental

evidence, i.e. no repetitions. Then the comparative

probabilities of a set of events may be taken to beproportional to the amount (from some fixed budget) one

would bet on each of the possible outcomes.


30/39

Repeatable Experiments

When there are independent repetitions along a singlehistorical sample path, it is possible to use the relative

frequency method of the standard repeatable experiment

model. Examples:

(i) yearly economic data,(ii) records of yearly hurricane data,

(iii) daily stock market price variations,

(iv) minute by minute repetitions of telephone call

requests and disconnections,

(v) second by second repetitions of packet transmission,

buffering and retransmissions through an Internet router.


31/39

Revision of Elements of Set Theory (|S| )

A set S is a collection of elements s, s, s,... for which

the subset relation is defined on S satisfying theaxioms:

1. {t, t, t, ..} S for any collection of elements of S,

which is written as t S in the case of a singleton.

2. The empty set satisfies S, and S S.

3. For any two sets A, B A B holds if and only if,

{s S; s A s B}.


32/39

Properties, Predicates and Sets

Let A satisfy A S, i.e. let A be a subset of S; thendefine the property, or predicate, PA at any t S via:

PA(t) { t is an element of A } { t A}

Then s satisfies PA , i.e. PA(s) is true, if and onlyif s A. Equivalently, PA(s) is true whenever s A andis false whenever s / A.

Example: For the integers Z, let Po(n) = { n Z, n odd}

Inter-relationships of sets can be defined and new sets

generated by specifications of properties.


33/39

Elementary Set Theoretic Relations

All sets below are subsets of S: A S, B S,etc

Def: A B {s S; s A s B}

{s S: PA(s) PB(s)}

PA PB

A B is of course also written as B A

Example: PA(s) s is a signal which passes throughbuffers B1 and B2; PB(s) s is a signal which passesthrough buffer B2. Clearly PA PB.

Def: A = B {s S; s A s B} and so


34/39

A = B { PA PB} A B and B A

Def: A B PAB {s S; s A or s B}

{s S; PA(s) PB(s)}

PA PB

Def: A B {s S; s A and s B}

Def: Ac {s S; s / A}


35/39

Properties and Operations on Sets

The following may be verified using the basic definitions

of the subset relation and the complement property (with

respect to the set S containing all the other sets.)

A = B {s A s B} and so

A = B A B and B A

A B Ac Bc where A, B S

A A S

A A A S

Ac, Ac c S; (complements wrt S)

so, since S c, the two way inclusion gives c = S


36/39

Commutative Properties of and :

A B = B A & A B = B A

Associative Property of and :

A (B C) = (A B) C = A B C

A (B C) = (A B) C = A B C

Distributive Property of and :

A (B C) = (A B) (A C)

A (B C) = (A B) (A C)

All may be clearly expressed in terms of Venn diagrams


37/39


38/39

De Morgans Rules:

(A B)c = Ac Bc

(A B)c

= A

c

Bc


39/39

Generalization to N Sets

(

Ni=1Ei)c =

Ni=1Eci

(N

i=1Ei)c =

Ni=1E

ci

Documents

01 Principles of Probablity 1