Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Frequency Distribution and
Summary Statistics
Dongmei Li
Department of Public Health Sciences Office of Public Health Studies University of Hawai’i at Mānoa
Outline
1. Stemplot
2. Frequency table
3. Summary statistics
2
1. Stem-and-leaf plots (stemplots)
Always start by looking at the data with
graphs and plots
Our favorite technique for looking at a
single variable is the stemplot
A stemplot is a graphical technique that
organizes data into a histogram-like
display
You can observe a lot by looking – Yogi Berra
3
Stemplot Illustrative Example
Select an SRS of 10 ages
List data as an ordered array
05 11 21 24 27 28 30 42 50 52
Divide each data point into a stem-value
and leaf-value
In this example the “tens place” will be
the stem-value and the “ones place” will
be the leaf value, e.g., 21 has a stem value
of 2 and leaf value of 1
4
Stemplot illustration (cont.)
Draw an axis for the stem-values:
0| 1| 2| 3| 4| 5| ×10 axis multiplier (important!)
Place leaves next to their stem value
21 plotted (animation)
1
5
Stemplot illustration continued …
Plot all data points and rearrange in rank order:
0|5
1|1
2|1478
3|0
4|2
5|02
×10
Here is the plot horizontally: (for demonstration purposes)
8
7
4 2
5 1 1 0 2 0
------------
0 1 2 3 4 5
------------
Rotated stemplot
6
Interpreting Stemplots Shape
◦ Symmetry
◦ Modality (number of peaks)
◦ Kurtosis (width of tails)
◦ Departures (outliers)
Location
◦ Gravitational center mean
◦ Middle value median
Spread
◦ Range and inter-quartile range
◦ Standard deviation and variance
7
Shape “Shape” refers to the pattern when
plotted
Here’s the silhouette of our data X
X
X X
X X X X X X
-----------
0 1 2 3 4 5
-----------
Consider: symmetry, modality, kurtosis 8
Shape: Idealized Density Curve A large dataset is introduced
An density curve is superimposed to better discuss shape
9
Symmetrical Shapes
10
Asymmetrical shapes
11
Modality (no. of peaks)
12
Kurtosis (width of tails)
Mesokurtic (medium) Platykurtic (flat)
Leptokurtic (steep)
skinny tails
fat tails
Kurtosis is not be easily judged by eye
13
Stemplot – Second Example
Data: 1.47, 2.06, 2.36, 3.43, 3.74, 3.78, 3.94, 4.42
Stem = ones-place
Leaves = tenths-place
Round to keep one digit
after decimal point
(e.g., 1.47 1.5)
Do not plot decimal
|1|5
|2|14
|3|4789
|4|4
(×1)
Shape: asymmetric, skewed to the left, unimodal, no outliers
14
Draw a stemplot using JMP
Analyze---Distribution---Data---Stem and
Leaf
15
Open the JMP data set
named
Stem_and_leaf_plot.jmp
Third Illustrative Example (n = 26)
Age data set from 26 subjects
{14, 17, 18, 19, 22, 22, 23, 24, 24, 26, 26, 27, 28,
29, 30, 30, 30, 31, 32, 33, 34, 34, 35, 36, 37, 38}
16
Data set:
Stem_and_leaf_plot_example2.jmp
Distribution of the age variable?
2. Frequency Table
Frequency = count
Relative frequency = proportion or %
Cumulative frequency % less than or equal to level
AGE | Freq Rel.Freq Cum.Freq.
------+-----------------------
3 | 2 0.3% 0.3%
4 | 9 1.4% 1.7%
5 | 28 4.3% 6.0%
6 | 37 5.7% 11.6%
7 | 54 8.3% 19.9%
8 | 85 13.0% 32.9%
9 | 94 14.4% 47.2%
10 | 81 12.4% 59.6%
11 | 90 13.8% 73.4%
12 | 57 8.7% 82.1%
13 | 43 6.6% 88.7%
14 | 25 3.8% 92.5%
15 | 19 2.9% 95.4%
16 | 13 2.0% 97.4%
17 | 8 1.2% 98.6%
18 | 6 0.9% 99.5%
19 | 3 0.5% 100.0%
------+-----------------------
Total | 654 100.0%
17
Frequency Table with Class Intervals
When data are sparse, group data into class intervals
Create 4 to 12 class intervals
Classes can be uniform or non-uniform
End-point convention: e.g., first class interval of
0 to 10 will include 0 but exclude 10 (0 to
9.99)
Talley frequencies
Calculate relative frequency
Calculate cumulative frequency
18
Class Intervals
Class Freq Relative
Freq. (%)
Cumulative
Freq (%)
0 – 9 1 10 10
10 – 19 1
20 – 29 4
30 – 39 1
40 – 49 1 10 80
50 – 59 2 20 100
Total 10 100 --
Uniform class intervals table (width 10) for data:
05 11 21 24 27 28 30 42 50 52
19
Histogram
0
1
2
3
4
5
0-9
10_1
9
20-2
9
30-3
9
40-4
9
50-5
9
Age Class
A histogram is a frequency chart for a quantitative measurement. Notice how the bars touch.
20
Bar Chart
0
50
100
150
200
250
300
350
400
450
500
Pre- Elem. Middle High
School-level
A bar chart with non-touching bars is reserved for categorical measurements and non-uniform
class intervals
21
3. Summary Statistics
Central location ◦Mean ◦Median ◦Mode
Spread ◦ Range and interquartile range (IQR) ◦ Variance and standard deviation
22
Location: Mean
“Eye-ball method” visualize where plot would balance
Arithmetic method = sum values and divide by n
8
7
4 2
5 1 1 0 2 0
------------
0 1 2 3 4 5
------------
^
Grav.Center
Eye-ball method around 25 to 30 (takes practice)
Arithmetic method mean = 290 / 10 = 29
23
Notation n sample size
X the variable (e.g., ages of subjects)
xi the value of individual i for variable X
sum all values (capital sigma)
Illustrative data (ages of participants):
21 42 5 11 30 50 28 27 24
52
n = 10
X = AGE variable
x1= 21, x2= 42, …, x10= 52
xi = x1 + x2 + … + x10= 21 + 42 + … + 52 = 290
24
Central Location: Sample Mean
“Arithmetic average”
Traditional measure of central location
Sum the values and divide by n
“xbar” refers to the sample mean
n
i
in xn
xxxn
x
1
1121
25
Example: Sample Mean
Ten individuals selected at random have the following ages:
21 42 5 11 30 50 28 27 24 52
Note that n = 10, xi = 21 + 42 + … + 52 = 290, and
0.29)290(10
11 i
xn
x
0 10 20 30 40 50 60
Mean = 29
The sample mean is the gravitational center of a distribution
26
Uses of the Sample Mean
The sample mean can be used to predict:
The value of an observation drawn at
random from the sample
The value of an observation drawn at
random from the population
The population mean
27
Population Mean
Same operation as sample mean except based on entire population (N ≡ population size)
Conceptually important
Usually not available in practice
Sometimes referred to as the expected value
i
ix
NN
x 1
28
Central Location: Median Ordered array:
05 11 21 24 27 28 30 42 50 52
When n is even, the median is the average of the
(n ÷2)th data and the (n ÷2+1)th data.
When n is odd, the median is the ((n+1) ÷2)th
data.
For illustrative data: n = 10 → the median falls
between 27 and 28=(27+28) ÷ 2 =27.5
05 11 21 24 27 28 30 42 50 52
median
Average the adjacent values: M = 27.5 29
More Examples of Medians
Example A: 2 4 6
Median = 4
Example B: 2 4 6 8
Median = 5 (average of 4 and 6)
Example C: 6 2 4
Median 2
(Values must be ordered first)
30
The Median is Robust
This data set has a mean of 1636:
1362 1439 1460 1614 1666 1792 1867
The median is 1614 in both instances, demonstrating its
robustness in the face of outliers.
The median is more resistant to skews and
outliers than the mean; it is more robust.
Here’s the same data set with a data entry error “outlier”
(highlighted). This data set has a mean of 2743:
1362 1439 1460 1614 1666 1792 9867
31
Mode
The mode is the most commonly encountered value in the dataset
This data set has a mode of 7 {4, 7, 7, 7, 8, 8, 9}
This data set has no mode {4, 6, 7, 8} (each point appears only once)
The mode is useful only in large data sets with repeating values
32
Comparison of Mean, Median, Mode
Note how the mean gets pulled toward
the longer tail more than the median
mean = median → symmetrical distrib
mean > median → positive skew
mean < median → negative skew
33
Spread: Quartiles
Two distributions can be quite
different yet can have the same
mean
This data compares particulate
matter in air samples (μg/m3) at
two sites. Both sites have a
mean of 36, but Site 1 exhibits
much greater variability. We
would miss the high pollution
days if we relied solely on the
mean.
Site 1| |Site 2
---------------
42|2|
8|2|
2|3|234
86|3|6689
2|4|0
|4|
|5|
|5|
|6|
8|6|
×10
34
Spread: Range
Range = maximum – minimum
Illustrative example:
Site 1 range = 68 – 22 = 46
Site 2 range = 40 – 32 = 8
Beware: the sample range will
tend to underestimate the
population range.
Always supplement the range
with at least one addition
measure of spread
Site 1| |Site 2
----------------
42|2|
8|2|
2|3|234
86|3|6689
2|4|0
|4|
|5|
|5|
|6|
8|6|
×10
35
Spread: Quartiles
Quartile 1 (Q1): cuts off bottom quarter of data = median of the lower half of the data set
Quartile 3 (Q3): cuts off top quarter of data = median of the upper half of the data set
Interquartile Range (IQR) = Q3 – Q1 covers the middle 50% of the distribution
05 11 21 24 27 28 30 42 50 52
Q1 median Q3
Q1 = 21, Q3 = 42, and IQR = 42 – 21 = 21 36
Quartiles (Tukey’s Hinges) – Example 2 Data are metabolic rates (cal/day), n = 7
When n is odd, include the median in both halves of the data set.
Bottom half: 1362 1439 1460 1614 which has a median of 1449.5 (Q1)
Top half: 1614 1666 1792 1867 which has a median of 1729 (Q3)
1362 1439 1460 1614 1666 1792 1867
median
37
Five-Point Summary
Q0 (the minimum)
Q1 (25th percentile)
Q2 (median)
Q3 (75th percentile)
Q4 (the maximum)
38
Boxplots 1. Calculate 5-point summary. Draw box from Q1
to Q3 w/ line at median
2. Calculate IQR and fences as follows:
FenceLower = Q1 – 1.5(IQR)
FenceUpper = Q3 + 1.5(IQR)
Do not draw fences
3. Determine if any values lie outside the fences
(outside values). If so, plot these separately.
4. Determine values inside the fences (inside values)
Draw whisker from Q3 to upper inside value.
Draw whisker from Q1 to lower inside value
39
Illustrative Example: Boxplot
1. 5 pt summary: {5, 21, 27.5, 42, 52};
box from 21 to 42 with line @
27.5
2. IQR = 42 – 21 = 21. FU = Q3 + 1.5(IQR) = 42 + (1.5)(21) =
73.5
FL = Q1 – 1.5(IQR) = 21 – (1.5)(21) = –
10.5
3. None values above upper fence
None values below lower fence
4. Upper inside value = 52
Lower inside value = 5
Draws whiskers
Data: 05 11 21 24 27 28 30 42 50 52
60
50
40
30
20
10
0
Upper inside = 52
Q3 = 42
Q1 = 21
Lower inside = 5
Q2 = 27.5
40
Illustrative Example: Boxplot 2 Data: 3 21 22 24 25 26 28 29 31 51
60
50
40
30
20
10
0
O utside value (51)
O utside value (3 )
Inside value (21)
Upper hinge (29)
Lower h inge (22)
Median (25 .5 )
Inside value (31)
1. 5-point summary: 3, 22, 25.5, 29, 51: draw box
2. IQR = 29 – 22 = 7 FU = Q3 + 1.5(IQR) = 29 + (1.5)(7) = 39.5 FL = Q1 – 1.5(IQR) = 22 – (1.5)(7) = 11.6
3. One above top fence (51) One below bottom fence (3)
4. Upper inside value is 31 Lower inside value is 21 Draw whiskers
41
Illustrative Example: Boxplot 3
Seven metabolic rates:
1362 1439 1460 1614 1666 1792 1867
7N =
Data source: Moore,
2000
1900
1800
1700
1600
1500
1400
1300
1. 5-point summary: 1362, 1449.5, 1614, 1729, 1867
2. IQR = 1729 – 1449.5 = 279.5
FU = Q3 + 1.5(IQR) = 1729 + (1.5)(279.5) = 2148.25
FL = Q1 – 1.5(IQR) = 1449.5 – (1.5)(279.5) = 1030.25
3. None outside
4. Whiskers end @ 1867 and 1362
42
Boxplots: Interpretation
Location ◦ Position of median
◦ Position of box
Spread ◦ Hinge-spread (IQR)
◦ Whisker-to-whisker spread
◦ Range
Shape ◦ Symmetry or direction of skew
◦ Long whiskers (tails) indicate leptokurtosis (Long tails?)
43
Side-by-side boxplots
Boxplots are especially useful when comparing groups
44
Spread: Standard Deviation
Most common
descriptive
measures of spread
Based on deviations
around the mean.
This figure
demonstrates the
deviations of two of
its values
This data set has a mean of 36.
The data point 33 has a deviation of 33 – 36 = −3.
The data point 40 has a deviation of 40 – 36 = 4.
45
Variance and Standard Deviation
xxiDeviation =
2
xxSSiSum of squared deviations =
1
2
n
SSsSample variance =
2ss Sample standard deviation =
46
Standard deviation (formula)
2
)(1
1xx
ns
i
Sample standard deviation s is the estimator of
population standard deviation .
Sum of Squares
47
Illustrative Example: Standard Deviation
Observation Deviations Squared deviations
36 36 36 = 0 02 = 0
38 38 36 = 2 22 = 4
39 39 36 = 3 32 = 9
40 40 36 = 4 42 = 16
36 36 36 = 0 02 = 0
34 34 36 = 2 (2)2 = 4
33 33 36 = 3 (3)2 = 9
32 32 36 = 4 (4)2 = 16
SUMS 0* SS = 58
2
xxixx
i
ix
* Sum of deviations always equals zero
48
Illustrative Example (cont.)
232)g/m( 286.8
18
58
1
n
SSs
32g/m 88.2286.8 ss
Sample variance (s2)
Standard deviation (s)
49
Interpretation of Standard Deviation
Measure spread (e.g., if group was s1 =
15 and group 2 s2 = 10, group 1 has
more spread, i.e., variability)
68-95-99.7 rule (next slide)
Chebychev’s rule (two slides hence)
50
68-95-99.7 Rule Normal Distributions Only!
68% of data in the range μ ± σ
95% of data in the range μ ± 2σ
99.7% of data in the range μ ± 3σ
Example. Suppose a variable has a Normal
distribution with μ = 30 and σ = 10. Then:
68% of values are between 30 ± 10 = 20 to 40
95% are between 30 ± (2)(10) = 30 ± 20 = 10
to 50
99.7% are between 30 ± (3)(10) = 30 ± 30 = 0
to 60 51
Chebychev’s Rule
All Distributions
Chebychev’s rule says that at least 75% of
the values will fall in the range μ ± 2σ (for
any shaped distribution)
Example: A distribution with μ = 30 and σ
= 10 has at least 75% of the values in the
range 30 ± (2)(10) = 10 to 50
52
Rules for Rounding
Carry at least four significant digits during
calculations.
Round at last step of operation
Always report units
Always use common sense and good judgment.
53
Choosing Summary Statistics
Always report a measure of central
location, a measure of spread, and the
sample size
Symmetrical mound-shaped
distributions report mean and
standard deviation
Odd shaped distributions report 5-
point summaries (or median and IQR)
54
Software and Calculators
Use software and calculators to check work.
55
Excel Data Analysis ToolPak
Data set: Boxplot.xlsx
Summary statistics using Excel
56
Excel Data Analysis ToolPak
Get summary statistics using Excel
57
Boxplot using Excel
Youtube links to get boxplot in Excel
http://www.youtube.com/watch?v=s8ZW
4PVarwE&feature=related
58
0
10
20
30
40
50
60
70
Female age Male age
Boxplot and summary statistics by
JMP
Data set:
Boxplot.jmp
What do you say
about the
comparison of
distributions of
ages for females
and males?
59
60
Exercise
Surgical times. Durations of surgeries (hours) for 15 patients receiving artificial hearts are shown here. Create a stem plot of these data. Describe the distribution. Are there any outliers? What is the standard deviation of this data set? Draw a boxplot based on this data set.
7.0 6.5 3.5 3.1 2.8 2.5 3.8 2.6 2.4 2.1 1.8 2.3 3.1 3.0 2.5
Data set: Presentation2_Exercise.xlsx
Presentation2_exercise.jmp 61