Basic Probability and Statistic

Editor: Nguyen Duc Minh KhoiEmail: ducminhkhoi@gmail.com

Website: https://nguyenducminhkhoi.blogspot.com

Probability: The basics

Probability: The basics (cont.)• Conditional Probability

• Independence

Probability: The basics (cont.)• Rule of total Probability:

• Bayes Rule:

• Chain rule

iiBAPBPAp |

)(

)|()(|

yP

xyPxPyxP

Probability: Random Variable (rv) – PMF vs. PDF• Discrete RV

• Continuous RV

Probability: Random Variable (rv) - Cumulative Density Functions (CDF)

Probability: Expectations

Probability: Expectations (cont.)

• Conditional expectation

Probability: Expectations (cont.)

• Other important Values

• Example:

Probability: Important discrete rv

Probability: Important continuous rv

Probability: Multiple variables

Probability: Multiple variables (Covariance & Correlation)

Law of large number & Central Limit Theorem

Statistics• Give data, how to find the model (pattern) of this

data.

• 2 school of thoughts:

• 𝜃in Bayesian is a rv (have prior p(𝜃)); 𝜃 in Classical is unknown constant.

Classical Inference: Maximum Likelihood Estimator (MLE)

Step: 1. Log; 2. derive; 3. solve for 𝜃

Classical Inference: Other methods• Linear Regression

Bayesian Inference:

Bayesian Inference (cont.)

References

• http://www.stanford.edu/class/cme308/OldWebsite/notes/chap2.pdf

• http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/resource-index/

• Bertsekas, Dimitri P. Introduction to Probability: Dimitri P. Bertsekas and John N. Tsitsiklis. Athena scientific, 2002.