Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
The Logic of Probability Theory
Deriving Bayesian Statistics from Boolean Algebra
William Sipes
July 14, 2010
Probability Theory as Extended Logic
• George Boole • The Laws of Thought • Algebraic expression of Aristotelian logical
propositions • Full Title: An Investigation of the Laws of Thought
on Which are Founded the Mathematical Theories of Logic and Probabilities (1854)
Probability Theory as Extended Logic
• Cox and Jaynes • The Algebra of Probable Inference (1961) • Probability Theory: The Logic of Science (2003) • Boolean Logic and Three Desiderata necessitate
Bayesian Probability
Boolean Algebra
• Commutative ring wrt operations of disjunction and conjunction
• Equivalence classes of [0] and [1] (representing FALSE and TRUE)
• Foundation of Computer Science
• Finite Field
Boolean Algebra
Though there are three distinct operations, it can be shown that any combination of two that includes negation is complete.
Disjunction
Conjunction
Negation
Bayesian Probability
• Conditional Probability • Differs from frequentist approaches • Based on prior distributions • Models can be updated with new information
The Desiderata
1) Representation of plausibility with real numbers
2) Qualitative correspondence with common sense
3) Structural Consistency
These uniquely determine the allowable operations of all probabilistic theory.
Assumption of twice differentiability in functions
Product Rule
1. Require that any plausibilities obey all of the desiderata simultaneously, unambiguously, and completely
2. Assume that there is a functional relation for conjoined propositions
3. Argue (using the requirements imposed by the desiderata) that there is only one form for this rule
4. Derive the form of the rule using differential equations
Product Rule
• Most basic assumption • Only functional form that does not degenerate
when tested at “extremes” • Key feature of familiar probability theory
Sum Rule
1. Using the product rule, derive a function that relates propositions and their negations
2. Impose the conditions of the desiderata to derive another functional equation
3. Argue analytically about the functional equation 4. Reduce the functional equation to a differential
equation 5. This leaves a functional relation for complementary
plausibilities