Random Sampling, Point Estimation and Maximum Likelihood

PowerPoint Presentation

Mathematical StatisticsRandom Sampling, Point Estimation and Maximum LikelihoodStatistical InferenceThe field of statistical inference consists of those methods used to make decisions or to draw conclusions about a population. These methods utilize the information contained in a sample from the population in drawing conclusions. Statistical inference may be divided into two major areas: Parameter Estimation Hypothesis TestingSamplingIf all members of a population are identical, the population is considered to be homogenous.

When individual members of a population are different from each other, the population is considered to be heterogeneous (having significant variation among individuals).

What is Sampling?Population SampleUsing data to say something (make an inference) with confidence, about a whole (population) based on the study of a only a few (sample).Sampling FrameSampling ProcessWhat you want to talk aboutWhat you actually observe in the dataInferenceEstimation of ParametersSample mean and sample variance are the two most important parameters of a sample. measures the central location of the sample values and s2 their spread (their variability).Small s2 may indicate high quality of production, high accuracy of measurement, etc.Note that and s2 will generally vary from sample to sample taken from the same population.Point Estimation of Parameters

A point estimate of a parameter is a number (point on real line: p in Binomial, and in Normal Distribution), which is computed from a given sample and serves as an approximation of the unknown exact value of the parameter of the population.Interval EstimateA point estimate is a statistic taken from a sample and is used to estimate a population parameter.

An Interval Estimate is an interval (Confidence Interval).

Point EstimatesEstimate PopulationParameters with SampleStatisticsMeanProportionVarianceDifference

Approximation of Mean

Method of MomentsKth Moment of a Samplex1, x2, x3,, xnLikelihood FunctionConsider a random variable X whose probability/ density f(x) depends on a single parameter :

Discrete Probability of n elements is l = f(x1) f(x2) f(x3)f(xn) where xj x xj + x, j = 1, 2, , nSince f(xj) depend on , the function l depends on x1, x2, x3,, xn (given and fixed) and .

Likelihood FunctionThe likelihood function is:

Likelihood Function is an approximation for the unknown value of for which l is as large as possible. If l is differentiable function of , a necessary condition for to have a maximum in an interval is:

Problem 1Find the maximum likelihood estimate for the parameter of a Normal distribution with known variance 2 = 02.Problem 3Derive the maximum likelihood estimate for the parameter p of a Binomial distribution. Problem 5Suppose that 4 times 5 trials were made and in the first 5 trials A happened 2, 1, 4, 4 times, respectively. Estimate p. Problem 7Consider X = number of independent trials until an event A occurs. Show that X has the probability function f(x) = pqx-1, x = 1, 2, , where p is probability of A in a single trial and q = 1 p. Find the maximum likelihood estimate for the parameter p corresponding to a single observation x of X.Problem 9Apply the maximum likelihood method to Poisson distribution. Problem 11Find the maximum likelihood estimate of in the density f(x) = e-x , if x 0 and f(x) = 0, if x < 0. Problem 13Compute ^ from the sample 1.8, 0.4, 0.8, 0.6, 1.4. Graph the Sample Distribution Function F^(x) and the Distribution Function F(x), with = ^ on the same axes. Do they agree reasonably well. 7-2 General Concepts of Point Estimation

7-2.1 Unbiased EstimatorsDefinition

7-2 General Concepts of Point Estimation

Example 7-1


Example 7-1 (continued)


7-2.3 Variance of a Point EstimatorDefinition

Figure 7-1 The sampling distributions of two unbiased estimators


7-2.3 Variance of a Point EstimatorTheorem 7-1


7-2.4 Standard Error: Reporting a Point EstimateDefinition


7-2.4 Standard Error: Reporting a Point Estimate


Example 7-2




7-2.6 Mean Square Error of an EstimatorDefinition


7-2.6 Mean Square Error of an Estimator


7-2.6 Mean Square Error of an Estimator

Figure 7-2 A biased estimator that has smaller variance than the unbiased estimator

7-3 Methods of Point Estimation

Definition

Definition


Example 7-4


7-3.2 Method of Maximum LikelihoodDefinition


Example 7-6




Figure 7-3 Log likelihood for the exponential distribution, using the failure time data. (a) Log likelihood with n = 8 (original data). (b) Log likelihood if n = 8, 20, and 40.7-3 Methods of Point Estimation

Example 7-9




Properties of the Maximum Likelihood Estimator


The Invariance Property


Example 7-10


Complications in Using Maximum Likelihood Estimation It is not always easy to maximize the likelihood function because the equation(s) obtained from dL()/d = 0 may be difficult to solve. It may not always be possible to use calculus methods directly to determine the maximum of L().7-3 Methods of Point Estimation

Example 7-11


Figure 7-4 The likelihood function for the uniform distribution in Example 7-11.7-4 Sampling Distributions

Statistical inference is concerned with making decisions about a population based on the information contained in a random sample from that population.

Definition

7-5 Sampling Distributions of Means

Theorem 7-2: The Central Limit Theorem


Figure 7-6 Distributions of average scores from throwing dice. [Adapted with permission from Box, Hunter, and Hunter (1978).]Example 7-13


Figure 7-7 Probability for Example 7-13.7-5 Sampling Distributions of Means

Definition

Point Estimation

Documents

Random Sampling, Point Estimation and Maximum Likelihood