CIS 2033A Modern Introduction to Probability
and StatisticsUnderstanding Why and How
Chapter 17: Basic Statistical Models
Slides by Dan Varano
Modified by Longin Jan Latecki
17.1 Random Samples and Statistical Models Random Sample: A random sample is a
collection of random variables X1, X2,…, Xn, that that have the same probability distribution and are mutually independent If F is a distribution function of each random
variable Xi in a random sample, we speak of a random sample from F. Similarly we speak of a random sample from a density f, a random sample from an N(µ, σ2) distribution, etc.
17.1 continued Statistical Model for repeated
measurements A dataset consisting of values x1, x2,…, xn of
repeated measurements of the same quantity is modeled as the realization of a random sample X1, X2,…, Xn. The model may include a partial specification of the probability distribution of each Xi.
17.2 Distribution features and sample statistics
Empirical Distribution Function Fn(a) =
Law of Large Numbers lim n->∞ P(|Fn(a) – F(a)| > ε) = 0
This implies that for most realizations Fn(a) ≈ F(a)
n
aX i ]),((#
17.2 cont. The histogram and kernel density estimate
≈ f(x) Height of histogram on (x-h, x+h] ≈ f(x) fn,h(x) ≈ f(x)
n
hxhxXi ]),((#
17.2 cont. The sample mean, sample median, and
empirical quantiles Ẋn ≈ µ
Med(x1, x2,…, xn) ≈ q0.5 = Finv(0.5)
qn(p) ≈ Finv(p) = qp
17.2 cont. The sample variance and standard
deviation, and the MAD Sn
2 ≈ σ2 and Sn ≈ σ
MAD(X1, X2,…,Xn) ≈ Finv(0.75) – Finv (0.5)
17.2 cont. Relative Frequencies
for a random sample X1,X2, . . . , Xn from a discrete distribution with probability mass function p,one has that ≈ p(a)n
aX i )(#
17.4 The linear regression model Simple Linear Regression Model: In a simple
linear regression model for a bivariate dataset (x1, y1), (x2, y2),…,(xn, yn), we assume that x1, x2,…, xn are nonrandom and that y1, y2,…, yn are realizations of random variables Y1, Y2,…, Yn satisfying Yi = α + βxi + Ui for i = 1, 2,…, n,
Where U1,…, Un are independent random variables with E[Ui] = 0 and Var(Ui) = σ2
17.4 cont Y1, Y2,…,Yn do not form a random sample.
The Yi have different distributions because every Yi has a different expectation E[Yi] = E[α + βxi + Ui] = α + βxi + E[Ui] = α + βxi