確率統計特論 (Probability and Statistics)tcs.inf.kyushu-u.ac.jp/~kijima/GPS20/GPS20-01.pdf確率統計特論(Probability and Statistics) May 13, 2020 来嶋秀治(Shuji Kijima)

確率統計特論 (Probability and Statistics)

May 13, 2020

来嶋秀治 (Shuji Kijima)

システム情報科学研究院情報学部門

Dept. Informatics, ISEE

Todays topics

• what is probability?

• probability space

確率統計特論(Probability & Statistics)

共通基礎科目

Preview

About this class

3

Evaluation

• exercises (レポート課題) =: 𝑧 (0 ≤ 𝑧 ≤ 10); optional

Almost every class (10回程度)

Submit from moodle before the final exam.

• midterm (中間試験): (mid) June, =:𝑥 (0 ≤ 𝑥 ≤ 50)

• final (期末試験): late July or early Aug., =:𝑦 (0 ≤ 𝑦 ≤ 50)

• score =𝑥 + 𝑦 (+𝑧).

• passing mark = 60. No makeup exam (追試は無い)

supplemental option for credit

not mandatory, but recommend.

4

Lecture

• Use moodle

• begin with S4B + slides, (teams, whiteboard, etc.)

• If you have any troubles, contact with chat/e-mail.

• contact: [email protected]

• Language: speak in Japanese, write in English.

• mathematical backgrounds (undergraduate level)

e.g., calculus (微積分), linear algebra (線形代数), etc…

• Back-up page

http://tcs.inf.kyushu-u.ac.jp/~kijima/

you can find exercise, slides, keywords, and schedule.

• Office hour: Wed(水曜日) 15:00—16:30

5

References

• 藤澤洋徳, 確率と統計, 現代基礎数学 13, 朝倉書店, 2006.

• 中田寿夫，内藤貫太，確率・統計，学術図書出版社，2017.

• M. Mitzenmacher, E. Upfel,

Probability and Computing: Randomized Algorithms and Probabilistic Analysis,

Cambridge University Press, 2005.

(邦訳) 小柴健史, 河内亮周, 確率と計算 --乱択アルゴリズムと確率的解析--,

共立出版, 2009.

[確率論]

• 伏見正則, 確率と確率過程, 朝倉書店, 2002.

[統計学]

• 日本統計学会, 統計学 (統計検定1級対応), 東京図書, 2013.

• 東大教養学部統計学教室編, 統計学入門, 東京大学出版会, 1991.

Check the web page!

http://www.tcs.inf.kyushu-u.ac.jp/~kijima/

Requirement (前提知識)6

Analysis (解析学)

• Differentiation (微分)

Ex. 1. d

d𝑥log 𝑥 = ?

• Integration by parts (部分積分)

Ex. 2. 0

3𝑥 exp(−𝑥) d𝑥 =?

Linear algebra (線形代数学)

• Bases (基底)

• Eigenvalues/diagonalization (固有空間/対角化)

What is Probability?

Examples

Ex 1. Monty Hall problem --- ask Marilyn8

You are given the choice of three doors:

Behind on door is a car; behind the others goats.

You pick a door, say “A”.

The host (Monty), who knows what's behind the doors,

opens another door, say “C”, which he knows has a goat.

He then says to you, “Do you want to pick door “B”?”

Question

Is it to your advantage to switch your choice?

図: wikipedia”モンティーホール問題”より

Explain in next time!

Hint: A modified Monty Hall problem --- ask Marilyn9

You are given the choice of 26 doors:

Behind on door is a car; behind the others goats.

You pick a door, say “A”.

The host (Monty), who knows what's behind the doors,

opens all other doors, say “C” to “Z”, except for “B”,

each of which he knows has a goat.

He then says to you, "Do you want to pick door “B”?”

Question

Is it to your advantage to switch your choice?

図: wikipedia”モンティーホール問題”より

Explain in next time!

Ex 2. Bertrand paradox10

Consider an equilateral triangle inscribed in a circle.

Suppose a chord of the circle is chosen at random.

Question

What is the probability that the chord is

longer than a side of the triangle?




Question


longer than a side of the triangle?

13

Without loss of generality,

assume that the radius of the circle is 1.

Then, the side length of equatorial

triangle in a circle (内接正三角形) is …

Answer 1: angle method.

Consider the “angle 𝜃” between the chord and the tangent line.

Note 0 ≤ 𝜃 ≤ 𝜋

The chord is longer than 3

iff𝜋

3≤ 𝜃 ≤

2𝜋

3

the probability is 1

3.




Question


longer than a side of the triangle (=:x)?




Question



Answer 2: level method.

Consider the “height ℎ” of the chord from the horizontal line.

Note 0 ≤ ℎ ≤ 2.


iff1

2≤ ℎ ≤

3

2

the probability is 𝟏

𝟐.

ℎ




Question



Answer 3: distance from the center.

Consider the “distance 𝑑” from the center of the circle to the chord.

Note 0 ≤ 𝑑 ≤ 1.


Iff 0 ≤ 𝑑 ≤1

2(the chord touches the yellow circle)

the probability is 𝟏

𝟒.

(prob. that the midpoint of the chord is in the yellow circle)




Question



What does

“a chord of the circle is chosen at random”

mean?

Probability Space

Axiom and theorems

17

Example: Probability Space

Ex. 1. A fair coin

Tossing a coin.

Ω = ℎ, 𝑡

𝐹 = 2 ℎ,𝑡 = ∅, ℎ , 𝑡 , ℎ, 𝑡

𝑃 ∅ = 0, 𝑃 ℎ =1

2, 𝑃 𝑡 =

1

2, 𝑃 ℎ, 𝑡 = 1

A probability space is defined by (Ω, ℱ, 𝑃)

Ω: sample space(標本空間); a set of elementally events(標本点),

an event(事象) is a subset of Ω.

ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).

𝑃: probability measure(確率測度); a function ℱ → ℝ,

probability of an event.

18

Example: Probability Space

Ex. 2. A die

Ω = {1,2,3,4,5,6},

ℱ = 2Ω = 1 , 2 , … , 1,3,5 , … , 1,2,3,4,5,6 , ∅

𝑃 𝐴 =𝐴

6for any 𝐴 ⊆ Ω.







19

Definition: Probability Space

𝑃 is a probability measure (Kolmogorov axioms)

1. 𝑃 𝐴 ≥ 0 for any 𝐴 ∈ ℱ.

2. 𝑃 Ω = 1.

3. Any countable sequence of mutual exclusive events 𝐴1, 𝐴2, …

satisfies that 𝑃 𝐴1 ∪ 𝐴2 ∪⋯ = 𝑃 𝐴1 + 𝑃 𝐴2 +⋯.







20

Theorem: Property of Probability (1/2)

Kolmogolov’s axiom (P1), (P2), (P3) implies the following.

Thm. 1

Let Ω,ℱ, 𝑃 be an arbitrary probability space.

(1) 𝑃 ∅ = 0．

(2) If 𝐴 ⊆ 𝐵 (𝐴, 𝐵 ∈ ℱ), then 𝑃 𝐵 ∖ 𝐴 = 𝑃 𝐵 − 𝑃 𝐴 .

(rem. 𝐵 ∖ 𝐴 = 𝐵 ∩ ഥ𝐴 )

(3) If 𝐴 ⊆ 𝐵 (𝐴, 𝐵 ∈ ℱ), then 𝑃 𝐴 ≤ 𝑃 𝐵 .

(4) 𝑃 𝐴 ≤ 1 for any 𝐴 ∈ ℱ.

(5) 𝑃 ഥ𝐴 = 1 − 𝑃(𝐴) for any 𝐴 ∈ ℱ.

21

Theorem: Property of Probability (2/2)

Thm. 1. (cont.)

(6) P 𝑃 𝐴 ∪ B = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴 ∩ 𝐵 for any 𝐴 ∈ ℱ.

(7) If 𝐴1, 𝐴2, … ∈ ℱ satisfy 𝐴1 ⊆ 𝐴2 ⊆ ⋯,

then 𝑃 𝑖𝐴𝑖ڂ = lim𝑛→∞

𝑃 𝐴𝑛

(8) If 𝐴1, 𝐴2, … ∈ ℱ satisfy 𝐴1 ⊇ 𝐴2 ⊇ ⋯,

then 𝑃 𝑖𝐴𝑖ځ = lim𝑛→∞

𝑃 𝐴𝑛

(9) For any 𝐴1, 𝐴2, … ∈ ℱ,

𝑃 𝑖𝐴𝑖ڂ = 0 ⇔ ∀𝑖 ≥ 0, 𝑃 𝐴𝑖 = 0

Kolmogolov’s axiom (P1), (P2), (P3) implies the following.

Proof of (1): 𝑃 ∅ = 0. 22

(1) 𝑃 ∅ = 0.

Proof:

Proof of (1): 𝑃 ∅ = 0. 23

(1) 𝑃 ∅ = 0.

Proof: 𝑃 Ω = 1 (∗) by Axiom (2).

𝑃 Ω = 𝑃 Ω ∪ ∅ (𝑏𝑦 𝑠𝑒𝑡 𝑡ℎ𝑒𝑜𝑟𝑦)

= 𝑃 Ω + 𝑃 ∅ 𝑏𝑦 𝐴𝑥𝑖𝑜𝑚 3

= 1 + 𝑃 ∅ ∗∗ 𝑏𝑦 𝐴𝑥𝑖𝑜𝑚 2 .

Thus, 1 = 1 + 𝑃 ∅ ∗∗∗ 𝑏𝑦 ∗ 𝑎𝑛𝑑 ∗∗ .

Finally, 𝑃 ∅ = 0 𝑏𝑦 ∗∗∗ 𝑎𝑛𝑑 𝑎𝑟𝑖𝑡ℎ𝑚𝑒𝑡𝑖𝑐 𝑜𝑓 𝑟𝑒𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 .

Every “reason” must be found in

Axiom (and set theory, arithmetic of reals)

and propositions already proved.

Let’s prove (4): ∀𝐴 ∈ ℱ, 𝑃 𝐴 ≤ 1.24

Firstly, we prove (3).

(3) If 𝐴 ⊆ 𝐵 then 𝑃 𝐴 ≤ 𝑃 𝐵 .

Proof:




Let’s prove (4): ∀𝐴 ∈ ℱ, 𝑃 𝐴 ≤ 1.25

Firstly, we prove (3).

(3) If 𝐴 ⊆ 𝐵 then 𝑃 𝐴 ≤ 𝑃 𝐵 .

Proof: 𝑃 𝐵 = 𝑃 𝐵 ∖ 𝐴 ∪ 𝐴 = 𝑃 𝐵 ∖ 𝐴 + 𝑃 𝐴 (*) by Axiom (3).

𝑃 𝐵 ∖ 𝐴 ≥ 0 (**) by Axiom(1).

𝑃 𝐵 ≥ 𝑃 𝐴 by (*), (**) and arithmetic of reals.

(4) ∀𝐴 ∈ ℱ, 𝑃 𝐴 ≤ 1.

Proof. 𝐴 ⊆ Ω since 𝐴 ∈ ℱ. Thus,

𝑃 𝐴 ≤ 𝑃 Ω 𝑏𝑦 3 𝑜𝑓 𝐿𝑒𝑚𝑚𝑎

= 1 𝑏𝑦 𝐴𝑥𝑖𝑜𝑚 2 .




Appx: Integration by parts (部分積分)26

න𝑎

𝑏

𝑥 exp(−𝑥) d𝑥 = −𝑥 exp −𝑥 𝑎𝑏 −න

𝑎

𝑏

− exp −𝑥 d𝑥

= ൫−𝑏 exp −𝑏 + 𝑎 exp −𝑎

Thm.

𝑓 𝑥 𝑔 𝑥 d𝑥 = 𝑓 𝑥 𝐺 𝑥 − 𝑓′ 𝑥 𝐺 𝑥 𝑑𝑥

27

Appx: -algebra

ℱ is a -algebra

1. ℱ contains the empty set: ∅ ∈ ℱ

2. ℱ is closed under complements: 𝐴 ∈ ℱ ഥ𝐴 ∈ ℱ

3. ℱ is closed under countable unions: 𝐴𝑖 ∈ ℱ 𝑖𝐴𝑖ڂ ∈ ℱ







28

Appx: Probability Space

Ex. 2. tossing 2 fair coins

Tossing 2 coins, which look alike. We can observe only

how many heads or tails are.

Ω = ℎ, ℎ , ℎ, 𝑡 , 𝑡, ℎ , 𝑡, 𝑡 (why?)

𝐹 = 2 0,1,2 (why?)

𝑃 ∅ = 0, 𝑃 0 = 1/4 , 𝑃 1 = 2/4, 𝑃 {0,1} = 3/4, …







29

Appx: Probability Space

Ex. 2. tossing 3 fair coins

Tossing 3 coins, which look alike. We can observe only

how many heads or tails are.

Ω =ℎ, ℎ, ℎ , ℎ, ℎ, 𝑡 , ℎ, 𝑡, ℎ , ℎ, 𝑡, 𝑡 ,𝑡, ℎ, ℎ , 𝑡, ℎ, 𝑡 , 𝑡, 𝑡, ℎ , (𝑡, 𝑡, 𝑡)

(why?)

𝐹 = 2 0,1,2,3 (why?)

𝑃 ∅ = 0, 𝑃 0 = 1/8 , 𝑃 1 = 3/8, 𝑃 #ℎ is odd = 1/2, …







Documents

確率統計 特論 (Probability and Statistics)tcs.inf.kyushu-u.ac.jp/~kijima/GPS20/GPS20-01.pdf確率統計 特論(Probability and Statistics) May 13, 2020 来嶋秀治(Shuji Kijima)

確率統計特論 (Probability and Statistics)tcs.inf.kyushu-u.ac.jp/~kijima/GPS20/GPS20-01.pdf確率統計特論(Probability and Statistics) May 13, 2020 来嶋秀治(Shuji Kijima)