1. Introduction to Probability In some areas, such as mathematics or logic, results of some process can be known with certainty (e.g., 2+3=5). Most real life situations, however, involve variability and uncertainty. For example, it is uncertain whether it will rain tomorrow; the price of a given stock a week from today is uncertain Note_1 ; the number of claims that a car insurance policy holder will make over a one-year period is uncertain. Uncertainty or "randomness" (meaning variability of results) is usually due to some mixture of two factors: (1) variability in populations consisting of animate or inanimate objects (e.g., people vary in size, weight, blood type etc.), and (2) variability in processes or phenomena (e.g., the random selection of 6 numbers from 49 in a lottery draw can lead to a very large number of different outcomes; stock or currency prices fluctuate substantially over time).
Variability and uncertainty make it more difficult to plan or to make decisions. Although they cannot usually be eliminated, it is however possible to describe and to deal with variability and uncertainty, by using the theory of probability. This course develops both the theory and applications of probability.
It seems logical to begin by defining probability. People have attempted to do this by giving definitions that reflect the uncertainty whether some specified outcome or ``event" will occur in a given setting. The setting is often termed an ``experiment" or ``process" for the sake of discussion. To take a simple ``toy" example: it is uncertain whether the number 2 will turn up when a 6-sided die is rolled. It is similarly uncertain whether the Canadian dollar will be higher tomorrow, relative to the U.S. dollar, than it is today. Three approaches to defining probability are:
The classical definition: Let the sample space (denoted by ) be the set of all possible distinct outcomes to an experiment. The probability of some event is
provided all points in are equally likely. For example, when a die is rolled the probability of getting a 2 is because one of the six faces is a 2.
The relative frequency definition: The probability of an event is the proportion (or fraction) of times the event occurs in a very long (theoretically infinite) series of repetitions of an experiment or process. For example, this definition could be used to argue that the probability of getting a 2 from a rolled die is .
The subjective probability definition: The probability of an event is a measure of how sure the person making the statement is that the event will happen. For example, after considering all available data, a weather forecaster might say that the probability of rain today is 30% or 0.3.
Unfortunately, all three of these definitions have serious limitations.
What does "equally likely" mean? This appears to use the concept of probability while trying to define it! We could remove the phrase "provided all outcomes are equally likely", but then the definition would clearly be unusable in many settings where the outcomes in did not tend to occur equally often.
Relative Frequency Definition:
Since we can never repeat an experiment or process indefinitely, we can never know the probability of any event from the relative frequency definition. In many cases we can't even obtain a long series of repetitions due to time, cost, or other limitations. For example, the probability of rain today can't really be obtained by the relative frequency definition since today can't be repeated again.
This definition gives no rational basis for people to agree on a right answer. There is some controversy about when, if ever, to use subjective probability except for personal decision-making. It will not be used in Stat 230.
These difficulties can be overcome by treating probability as a mathematical system defined by a set of axioms. In this case we do not worry about the numerical values of probabilities until we consider a specific application. This is consistent with the way that other branches of mathematics are defined and then used in specific applications (e.g., the way calculus and real-valued functions are used to model and describe the physics of gravity and motion).
The mathematical approach that we will develop and use in the remaining chapters assumes the following:
probabilities are numbers between 0 and 1 that apply to outcomes, termed ``events'',
each event may or may not occur in a given setting.
Chapter 2 begins by specifying the mathematical framework for probability in more detail.
ExercisesTry to think of examples of probabilities you have encountered which might have been obtained by each of the three ``definitions".
Which definitions do you think could be used for obtaining the following probabilities?
You have a claim on your car insurance in the next year.
There is a meltdown at a nuclear power plant during the next 5 years.
A person's birthday is in April.
Give examples of how probability applies to each of the following areas.
Auditing of expense items in a financial statement
Disease transmission (e.g. measles, tuberculosis, STD's)
Public opinion polls
2. Mathematical Probability Models Sample Spaces and ProbabilityConsider some phenomenon or process which is repeatable, at least in theory, and suppose that certain events (outcomes) are defined. We will often term the phenomenon or process an ``experiment" and refer to a single repetition of the experiment as a ``trial". Then the probability of an event , denoted , is a number between 0 and 1.
If probability is to be a useful mathematical concept, it should possess some other properties. For example, if our ``experiment'' consists of tossing a coin with two sides, Head and Tail, then we might wish to consider the events = ``Head turns up'' and = ``Tail turns up''. It would clearly not be desirable to allow, say, and , so that . (Think about why this is so.) To avoid this sort of thing we begin with the following definition.
DefinitionA sample space is a set of distinct outcomes for an experiment or process, with the property that in a single trial, one and only one of these outcomes occurs. The outcomes that make up the sample space are called sample points.
A sample space is part of the probability model in a given setting. It is not necessarily unique, as the following example shows.
Example: Roll a 6-sided die, and define the events
Then we could take the sample space as . However, we could also define events even number turns up odd number turns up
and take . Both sample spaces satisfy the definition, and which one we use would depend on what we wanted to use the probability model for. In most cases we would use the first sample space.
Sample spaces may be either discrete or non-discrete; is discrete if it consists of a finite or countably infinite set of simple events. The two sample spaces in the preceding example are discrete. A sample space consisting of all the positive integers is also, for example, discrete, but a sample space consisting of all positive real numbers is not. For the next few chapters we consider only discrete sample spaces. This makes it easier to define mathematical probability, as follows.
DefinitionLet be a discrete sample space. Then probabilities are numbers attached to the 's such that the following two conditions hold:
The set of values is called a probability distribution on .
DefinitionAn event in a discrete sample space is a subset If the event contains only one point, e.g. we call it a simple event. An event made up of two or more simple events such as is called a compound event.
Our notation will often not distinguish between the point and the simple event which has this point as its only element, although they differ as mathematical objects. The condition (2) in the definition above reflects the idea that when the process or experiment happens, some event in must occur (see the definition of sample space). The probability of a more general event (not necessarily a simple event) is then defined as follows:
DefinitionThe probability of an event is the sum of the probabilities for all the simple events that make up .
For example, the probability of the compound event The definition of probability does not say what numbers to assign to the simple events for a given setting, only what properties the numbers must possess. In an actual situation, we try to specify numerical values that make the model useful; this usually means that we try to specify numbers that are consistent with one or more of the empirical ``definitions'' of Chapter 1.
Example: Suppose a 6-sided die is rolled, and let the sample space be , where means the number 1 occurs, and so on. If the die is an ordinary one, we would find it useful to define probabilities as
because if the die were tossed repeatedly (as in some games or gambling situations) then each number would occur close to of the time. However, if the die were weighted in some way, these numerical values would not be so useful.
Note that if we wish to consider some compound event, the probability is easily obtained. For example, if = ``even number" then because we get .
We now consider some additional examples, starting with some simple ``toy" problems involving cards, coins and dice and then considering a more scientific example.
Remember that in using probability we are actually constructing mathematical models. We can approach a given problem by a series of three steps:
Specify a sample space .