History of the Normal Distribution - University of Utahkenkel/normaldistributiontalk.pdf · Normal Distribution Jenny Kenkel The Normal Distribution The normal distribution is a family

History of theNormal

Distribution

Jenny Kenkel

History of the Normal Distribution

Jenny Kenkel

November 8th, 2016


Distribution

Jenny Kenkel

Probabiltiy Distributions

A probability distribution is a function f (x) so that

P(a < X < b) =

∫ b

af (x)dx


Distribution

Jenny Kenkel

The Normal Distribution

The normal distribution is afamily of distributions, given by

f (x) =1√

2πσ2e−

(x−µ)2

2σ2

The Standard Normal has µ =0 and σ = 1, i.e.

f (x) =1√2π

ex2

2

Changing µ changes the loca-tion of the curve, and chang-ing σ changes the spread of thecurve


Distribution

Jenny Kenkel

Arbuthnot: “An argument for Divine Providence, taken from

the constant Regularity observ’d in the Births of both Sexes”

Argued the probabilitythat more males thanfemales were born everyyear from 1629 to 1710,

82 years, was(

12

)82

But the fact that notmany more males thanfemales were born everyyear was proof polygamy iscontrary to the laws ofnature


Distribution

Jenny Kenkel

Arbuthnot: “An argument for Divine Providence, taken from

the constant Regularity observ’d in the Births of both Sexes”

Argued the probabilitythat more males thanfemales were born everyyear from 1629 to 1710,

82 years, was(

12

)82

But the fact that notmany more males thanfemales were born everyyear was proof polygamy iscontrary to the laws ofnature


Distribution

Jenny Kenkel

‘sGravesande

Felt evidence for divineprovidence was evenstronger than Arbuthnothad argued


Distribution

Jenny Kenkel

Bernoulli Trials

A Bernoulli Trial is an experiment with only two possibleoutcomes. A Binomial Experiment is a series of repeatedBernoulli trials.

The probability, p, of success stays constant as more trialsare performed

The probability of k successes in n trials is(n

k

)pk(1− p)n−k


Distribution

Jenny Kenkel

Bernoulli Trials




k

)pk(1− p)n−k


Distribution

Jenny Kenkel

Bernoulli Trials




k

)pk(1− p)n−k


Distribution

Jenny Kenkel

‘sGravesande was very patient

‘sGravesande let n = 11, 429, p = 0.5

The lowest ratio, which occurred in 1703, was .5027,which corresponded to about 5745 out of 11429

The highest ratio, which occurred in 1661, was .5362,which corresponded to about 6128 out of 11429

‘sGravesande calculated

P[5745 ≤ x ≤ 6128] =6128∑5745

(11429

x

)(1

2

)11429

≈ 0.292

Thus the probability of this difference in birth ratesrecurring 82 years in a row is .29282


Distribution

Jenny Kenkel






P[5745 ≤ x ≤ 6128] =6128∑5745

(11429

x

)(1

2

)11429

≈ 0.292



Distribution

Jenny Kenkel






P[5745 ≤ x ≤ 6128] =6128∑5745

(11429

x

)(1

2

)11429

≈ 0.292



Distribution

Jenny Kenkel






P[5745 ≤ x ≤ 6128] =6128∑5745

(11429

x

)(1

2

)11429

≈ 0.292



Distribution

Jenny Kenkel






P[5745 ≤ x ≤ 6128] =6128∑5745

(11429

x

)(1

2

)11429

≈ 0.292



Distribution

Jenny Kenkel

DeMoivre and Stirling’s Approximation

In 1733, in a second supplement that only appeared withcertain editions of his book[3], De Moivre noted that:

n! ∼ Bnn+ 12 e−n

James Stirling determined that B =√

2π


k

)pk(1− p)n−k ≈ 1√

2πnp(1− p)e− (k−np)2

2np(1−p)

Let σ2 = np(1− p) and µ = np, and you have(n

k

)pk(1− p)n−k ≈ 1√

2πσ2e−

(k−µ)2

σ2


Distribution

Jenny Kenkel



n! ∼ Bnn+ 12 e−n


2π


k

)pk(1− p)n−k ≈ 1√


2np(1−p)


k

)pk(1− p)n−k ≈ 1√

2πσ2e−

(k−µ)2

σ2


Distribution

Jenny Kenkel



n! ∼ Bnn+ 12 e−n


2π


k

)pk(1− p)n−k ≈ 1√


2np(1−p)


k

)pk(1− p)n−k ≈ 1√

2πσ2e−

(k−µ)2

σ2


Distribution

Jenny Kenkel



n! ∼ Bnn+ 12 e−n


2π


k

)pk(1− p)n−k ≈ 1√


2np(1−p)


k

)pk(1− p)n−k ≈ 1√

2πσ2e−

(k−µ)2

σ2


Distribution

Jenny Kenkel

From Binomials to Errors

In 1733, DeMoivre first used the Normal distribution as anapproximation for probabilities of binomial experiments where nis very large.About 70 years later, it would be used as the probabilitydistribution of random errors.


Distribution

Jenny Kenkel

The beginning of the average

Astronomy called for accurate measurements

In the 1600s, Tycho Brahe advocated incorporatingrepeated measurements but did not specify how to userepeated measurements

Sometimes astronomers used median observations,sometimes mean observations, some times ????


Distribution

Jenny Kenkel



In the 1600s, Tycho Brahe advocated incorporatingrepeated measurements but did not specify how to userepeated measurementsSometimes astronomers used median observations,sometimes mean observations, some times ????


Distribution

Jenny Kenkel



In the 1600s, Tycho Brahe advocated incorporatingrepeated measurements but did not specify how to userepeated measurementsSometimes astronomers used median observations,sometimes mean observations, some times ????


Distribution

Jenny Kenkel

Early reasoning about the probability distributionsof errors

In 1632, Galileo reasoned that

1 There is a true distance, which is only one number

2 All observations have errors

3 The errors are symmetric around the true value

4 Small errors are more common than large ones


Distribution

Jenny Kenkel








Distribution

Jenny Kenkel








Distribution

Jenny Kenkel








Distribution

Jenny Kenkel

The average didn’t take on immediately

In the 1660s, Robert Boyleargued that, rather thanaveraging at all, astronomersshould just focus on one verycareful experiment

In 1756, Simpson wrote that:“some persons, of considerablenote, have been of opinion, andeven publickly maintained, thatone single observation, takenwith due care, was as much tobe relied on as the Mean of agreat number ”


Distribution

Jenny Kenkel

The average didn’t take on immediately

In the 1660s, Robert Boyleargued that, rather thanaveraging at all, astronomersshould just focus on one verycareful experiment

In 1756, Simpson wrote that:“some persons, of considerablenote, have been of opinion, andeven publickly maintained, thatone single observation, takenwith due care, was as much tobe relied on as the Mean of agreat number ”


Distribution

Jenny Kenkel

Simpson’s probability distributions of errors!

Also in Simpson’s 1756 paper was the notion of probabilitydistributions of errors or error curve

Simpson computed the probabilities when the errors couldtake on values

−v , . . . ,−2,−1, 0, 1, 2, . . . , v

to have probabilities proportional to either

r−v , . . . , r−2, r−1, r0, r1, r2, . . . , r4

or

r−v , 2r1−v , 3r2−v . . . , (v + 1)r0, . . . , 2r v−1, r4


Distribution

Jenny Kenkel

Simpson’s probability distributions of errors!

Also in Simpson’s 1756 paper was the notion of probabilitydistributions of errors or error curveSimpson computed the probabilities when the errors couldtake on values

−v , . . . ,−2,−1, 0, 1, 2, . . . , v

to have probabilities proportional to either

r−v , . . . , r−2, r−1, r0, r1, r2, . . . , r4

or

r−v , 2r1−v , 3r2−v . . . , (v + 1)r0, . . . , 2r v−1, r4


Distribution

Jenny Kenkel

Laplace’s Error Curves

In 1774, Laplace proposed the first of his probabilitydistributions for errors .

Probability distribution, φ, must be symmetric about 0

Probability distribution, φ, must be monotone decreasingwith x“as we have no reason to suppose a different law for theordinates than for their differences...”, assumedφ(x) ∼ dφ(x)

dx ,

Thus φ(x) = m2 e−m|x |

Recall:

f (x) =1√

2πσ2e−

(x−µ)2

2σ2


Distribution

Jenny Kenkel



Probability distribution, φ, must be symmetric about 0Probability distribution, φ, must be monotone decreasingwith x

“as we have no reason to suppose a different law for theordinates than for their differences...”, assumedφ(x) ∼ dφ(x)

dx ,


Recall:

f (x) =1√

2πσ2e−

(x−µ)2

2σ2


Distribution

Jenny Kenkel



Probability distribution, φ, must be symmetric about 0Probability distribution, φ, must be monotone decreasingwith x“as we have no reason to suppose a different law for theordinates than for their differences...”, assumedφ(x) ∼ dφ(x)

dx ,


Recall:

f (x) =1√

2πσ2e−

(x−µ)2

2σ2


Distribution

Jenny Kenkel



Probability distribution, φ, must be symmetric about 0Probability distribution, φ, must be monotone decreasingwith x“as we have no reason to suppose a different law for theordinates than for their differences...”, assumedφ(x) ∼ dφ(x)

dx ,


Recall:

f (x) =1√

2πσ2e−

(x−µ)2

2σ2


Distribution

Jenny Kenkel

Gauss gets involved

On January 1, 1801, Giuseppe Piazzi spotted something hebelieved to be a new planet and named Ceres.


Distribution

Jenny Kenkel

Gauss gets involved

Six weeks later, he lost it!


Distribution

Jenny Kenkel

Gauss gets involved

Gauss suggested looking in a different area of the sky thanmost other astronomers... and he was right!


Distribution

Jenny Kenkel

Gauss’ Proof: Assumptions and Notation

Gauss concluded that the probability density for the error is:

φ(x) =h√πe−h

2x2

Let p be the true but unknown error

Let M1, . . . ,Mn be estimates of p

Let φ(x) be the probability density function of the randomerror(recall: P(a < x < b) <

∫ b

aφ(x) )

φ(x) has a maximum at x = 0

φ(−x) = φ(x)

The average, M = 1n

∑ni=1 Mi , is the most likely value of p


Distribution

Jenny Kenkel

Gauss’ Proof: Part 1 of 4

Let f (x) = φ′(x)φ(x)

Then f (−x) = −f (x)

If p is the true value and Mi is the i th estimate, thenMi − p is the i th error

If the errors are independent, then their joint probabilitydistribution is

Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

If M = 1n

∑ni=1 Mi is the most likely value of p, then M

should maximize Ω

That is, dΩdp |M = 0


Distribution

Jenny Kenkel






Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

If M = 1n


should maximize Ω



Distribution

Jenny Kenkel






Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

If M = 1n


should maximize Ω



Distribution

Jenny Kenkel






Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

If M = 1n


should maximize Ω



Distribution

Jenny Kenkel






Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

If M = 1n


should maximize Ω



Distribution

Jenny Kenkel






Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

If M = 1n


should maximize Ω



Distribution

Jenny Kenkel


Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

so 0 = dΩdp |p=M =

= −φ′(M1 −M)φ(M2 −M) · · ·φ(Mn −M)

− φ(M1 −M)φ′(M2 − p) · · ·φ(Mn −M)

− . . .− φ(M1 −M)φ(M2 −M) · · ·φ′(Mn −M)

=

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω


Distribution

Jenny Kenkel


Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

so 0 = dΩdp |p=M =

= −φ′(M1 −M)φ(M2 −M) · · ·φ(Mn −M)

− φ(M1 −M)φ′(M2 − p) · · ·φ(Mn −M)

− . . .− φ(M1 −M)φ(M2 −M) · · ·φ′(Mn −M)

=

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω


Distribution

Jenny Kenkel


Ω = φ(M1 − p)φ(M2 − p) · · ·φ(Mn − p)

so 0 = dΩdp |p=M =

= −φ′(M1 −M)φ(M2 −M) · · ·φ(Mn −M)

− φ(M1 −M)φ′(M2 − p) · · ·φ(Mn −M)

− . . .− φ(M1 −M)φ(M2 −M) · · ·φ′(Mn −M)

=

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω


Distribution

Jenny Kenkel


0 =

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω

Recall, we defined f (x) = φ′(x)φ(x) , so

0 = f (M1 −M) + . . .+ f (Mn −M)

For arbitrary real numbers M and N, let M1 = M, andM2 = M3 = . . . = Mn = M − nN.Then M = M − (n − 1)N

0 = f ((n− 1)N) + (n− 1)f (−N) or f ((n− 1)N) = (n− 1)f (N)


Distribution

Jenny Kenkel


0 =

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω


0 = f (M1 −M) + . . .+ f (Mn −M)


0 = f ((n− 1)N) + (n− 1)f (−N) or f ((n− 1)N) = (n− 1)f (N)


Distribution

Jenny Kenkel


0 =

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω


0 = f (M1 −M) + . . .+ f (Mn −M)

For arbitrary real numbers M and N, let M1 = M, andM2 = M3 = . . . = Mn = M − nN.

Then M = M − (n − 1)N

0 = f ((n− 1)N) + (n− 1)f (−N) or f ((n− 1)N) = (n− 1)f (N)


Distribution

Jenny Kenkel


0 =

(φ′(M1 −M)

φ(M1 −M) +

φ′(M2 −M)

φ(M2 −M) + . . .+

φ′(Mn −M)

φ(Mn −M)

)Ω


0 = f (M1 −M) + . . .+ f (Mn −M)


0 = f ((n− 1)N) + (n− 1)f (−N) or f ((n− 1)N) = (n− 1)f (N)


Distribution

Jenny Kenkel

Gauss’ Proof: Part 4

f ((n − 1)N) = (n − 1)f (N)

If f is continuous, then f (x) = kx

Thus, φ′(x)φ(x) = kx

Inegration with respect to x gives us

ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2

In order for φ(x) to have a maximum at x = 0, we must havethat k is negative, so set k = −h2.We require that

∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π

Thus φ(x) = h√πekx

2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2


∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2


∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2


∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2


∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2

In order for φ(x) to have a maximum at x = 0, we must havethat k is negative, so set k = −h2.

We require that∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2


∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel


f ((n − 1)N) = (n − 1)f (N)




ln(φ(x)) =k

2x2 + c

φ(x) = Aekx2

2


∫∞−∞ Ae−h

2x2dx = 1, so A = h√

π


2


Distribution

Jenny Kenkel

Not Just For Errors Anymore

In 1846, Adolphe Quetelet examined chest measurementsof Scottish soldiers from a medical journal.

He realized that measurements of different objects of thesame type would also often take on the NormalDistribution.

His idea was that individuals were errors from some true,abstract form: the average man.


Distribution

Jenny Kenkel

The Average Man

“The more knowledge becomes widespread, the moredo the deviations from the average diminish, and themore do we tend to approach that which is beautifuland that which is good.”

Quetelet, 1835


Distribution

Jenny Kenkel

Florence Nightingale

Mentored by Quetelet,translated one of his worksto English

Popularized the pie chart,invented the circularhistogram

Was the first femalemember of the RoyalStatistical Society


Distribution

Jenny Kenkel

Galton

Quetelet’s discoverybecame very popular inthe social sciences

Galton noticed the normalcurve appearing in heights,chest measurements, examscores, weight of sweetpeas

Galton was one of the firststatisticians to use theterm “normal” to describethe curve outlined by DeMoivre and Gauss


Distribution

Jenny Kenkel

Galton


Distribution

Jenny Kenkel

Pearson

In 1893, Pearson called the curve’ the normal curve(rather than the Gaussian).

He was trying to give some credit to Laplace, rather thanGauss- not even realizing that DeMoivre had discovered itearlier!

In 1900, Pearson developed a method of testing goodnessof fit


Distribution

Jenny Kenkel

Pearson





Distribution

Jenny Kenkel

Pearson





Distribution

Jenny Kenkel

Value at Risk Model

Value at Risk is a way of quantifying risk of investmentinto a single number

For example, if a VaR of 5% is $1 million, that means theprobability the portfolio value will decrease more than $ 1million is 5%

The original VaR measurement used the normaldistribution to model amount a portfolio would gain or lose

This assumption contributed to the housing crisis.


Distribution

Jenny Kenkel

Everyone Loves The Normal Distribution


Distribution

Jenny Kenkel

Anders Hald. History of Probability and Statistics andTheir Applications before 1750. John Wiley & Sons, Inc.,New Jersey, 2003.

Anders Hald. A History of Mathematical Statistics From1750 to 1930. John Wiley & Sons, Inc., New York, 1998.

Norman L. Johnson and Samual Kotz, editors. LeadingPersonalities in Statistical Sciences from teh seventeenthcentury to the present. John Wiley & Sons, Inc., NewYork, 1997.

Stephen M. Stigler. The History of Statistics: TheMeasurement of Uncertainty before 1900. The BelknapPress of Harvard University Press, Massachusetts. 1986.

Saul Stahl. The Evolution of the Normal Distribution.Mathematics Magazine, 96-113.

Documents

History of the Normal Distribution - University of Utahkenkel/normaldistributiontalk.pdf · Normal Distribution Jenny Kenkel The Normal Distribution The normal distribution is a family