Alpha Diversity Indices James A. Danoff-Burg Dept. Ecol., Evol., & Envir. Biol. Columbia University

Alpha Diversity IndicesAlpha Diversity Indices

James A. Danoff-Burg

Dept. Ecol., Evol., & Envir. Biol.

Columbia University

Lecture 4 – Alpha Diversity Indices © 2003 Dr. James A. Danoff-Burg, [email protected]

Alpha Diversity IndicesAlpha Diversity Indices Q-Statistic Intro to Alpha Diversity Indices Simpson McIntosh Berger-Parker Shannon-Wiener Brillouin Jack-Knifing Diversity Indices Pielou’s Hierarchical Diversity Index

Week 1

Week 2


Diversity of DiversitiesDiversity of Diversities Difference between the diversities is usually one of

relative emphasis of two main envir. aspects Two key features

Richness Abundance – our emphasis today

Each index differs in the mathematical method of relating these features

One is often given greater prominence than the other Formulae significantly differ between indices


Diversity LevelsDiversity Levels Progress from local to regional levels

Point: diversity at a single point or microenvironment• Our emphasis thus far

Alpha: within habitat diversity• Usually consists of several subsamples in a habitat

Beta: species diversity along transects & gradients• High Beta indicates number of spp increases rapidly with

additional sampling sites along the gradient Gamma: diversity of a larger geographical unit (island) Epsilon: regional diversity


Q Statistic Introduction A bridge between the abundance models &

diversity indices Does not involve fitting a model

• as in the abundance models

Provides an indication of community diversity No weighting towards very abundant or rare species

• They are excluded from the analysis• Whittaker (1972) created earlier analysis including these

– Thereby more influenced by the few rare / abundant species

Proposed by Kempton & Taylor (1976)


Q Statistic Visually Measures “inter-quartile slope” on the cumulative

species abundance curve

250

200

150

100

50

010 100 1,000 10,000

Cumulative Species

Species Abundance

S = 250S/4 = 62.5

1st = 62.52nd = 1253rd = 187.5

R1 = 62.5 = 0.25*S

R2 = 187.5 = 0.75*S

Q = slope


Q StatisticQ Statistic Relationship to other indices

Similar to the value in the log series model Q = (0.371)(S*) /

Biases in Q May be biased in small samples

• Because we are including more of the rare and abundant species in the calculation


Calculating QCalculating Q- Worked Example #6- Worked Example #6

Assemble table with 3 columns # Individuals, # Species, Summed # species

Determine R1 and R2 R1 should be > or = 0.25 * S R2 should be > or = 0.75 * S

Calculate Q Q = [((nR1)/2) + nr + ((nR2)/2)] / [ln(R2/R1)]

• nR1 and nR2 = # species in each quartile class

nr = total number of species between the quartiles

• R1 and R2 = # of individuals at each quartile break point



Week 1

Week 2


Alpha Diversity IndicesAlpha Diversity Indices All based on proportional species abundances

Species abundance models have drawbacks• Tedious and repetitive• Problems if the data do not violate more than one model

– How to choose between?

Building upon the species abundance models Allows for formal comparisons between sites /

treatments


Alpha Diversity IndicesAlpha Diversity Indices “Heterogeneity Indices”

Consider both evenness AND richness Species abundance models only consider evenness

No assumptions made about species abundance distributions

Cause of distribution Shape of curve

“Non-parametric” Free of assumptions of normality


Two General CategoriesTwo General Categories Information Theory (complicated computation)

Diversity (or information) of a natural system is similar to info in a code or message

Examples: Shannon-Wiener and Brillouin Indices

Species Dominance Measures (simple comput.) Weighted towards abundance of the commonest

species Total species richness is downweighted relative to

evenness Examples: Simpson, McIntosh, and Berger-Parker



Week 1

Week 2


Simpson Index ValuesSimpson Index Values Derived by Simpson (1949) Basis

Probability of 2 individuals being conspecifics If drawn randomly from an infinitely large community

Summarized by letter D, 1-D, or 1/D D decreases with increasing diversity

• Can go from 1 – 30+• Probability that two species are conspecifics with diversity

1-D and 1/D increases with increasing diversity• 0.0 < 1-D < 1.0• 0.0 < 1/D < 10+


Simpson IndexSimpson Index Heavily weighted towards most abundant species

Less sensitive to changes in species richness Once richness > 10 underlying species abundance is

important in determining the index value Inappropriate for some models

• Log Series & Geometric Best for Log-Normal

• Possibly Broken Stick10000

1000

100

10

10 20 30

Number of Species

D value

Log Series

Log Normal Series

Broken Stick Series


Simpson IndexSimpson Index When would this weight towards most abundant

species be desired? Not just when the abundance model fits the Log-Normal

Conservation implications of index use?


Simpson Calculation – Simpson Calculation – Worked Example 9Worked Example 9

Calculate N and S Calculate D

D = (ni(ni-1)) / (N(N-1) Solve and then sum for all species in the sample

Calculate 1/D Increases with increasing diversity



Week 1

Week 2


McIntosh IndexMcIntosh Index Proposed by McIntosh (1967) Community is a point in an S dimensional

hypervolume whose Euclidean distance from the origin is a measure of diversity

Paraphrased from Magurran Origin is no diversity, distances from origin are more

diverse

Not strictly a dominance index Needs conversion to dominance index


McIntosh ManipulationsMcIntosh Manipulations Base calculations (U metric)

Strongly influenced by sample size

Conversion to a dominance measure (D) Use Dm for our class Makes value independent of sample size

Derive a simple evenness index using McIntosh Most often used contribution of McIntosh


McIntosh Calculation – McIntosh Calculation – Worked Example 10Worked Example 10

Base calculations U = (ni

2)

• ni = abundance of ith species

• Different from Magurran’s definition

Conversion to a dominance measure Dm = (N-U) / (N-N)

Derive evenness index Em = (N-U) / ((N-(N/S))



Week 1

Week 2


Berger-ParkerBerger-Parker Proposed by Berger and Parker (1970) and

developed by May (1975) Simple calculation = d Expresses proportional importance of most

abundant species


Berger-ParkerBerger-Parker Decreasing d values increasing diversity

Often use 1 / d• Increasing 1 / d increasing diversity• And reduction in dominance of one species

Independent of S, influenced by sample size Comparability between sites if sampling efforts

standardized

Question may lead to use of Berger-Parker Example: Change in dominant species in diet?


Berger-Parker – An ExampleBerger-Parker – An Example

River Mouth #1 Intertidal #2 Intertidal #3 + Sewage #4 Fresh & Hot #5

Nereis 394 1642 90 126 32

Corophium 3487 5681 320 17 0

Gammarus 275 196 180 115 0

Tubifex 683 1348 46 436 5

Chironomids 22 12 2 27 0

Insect larvae 1 0 0 0 0

Arachnid 0 1 0 0 0

Carcinus 4 48 1 3 0

Cragnon 6 21 0 1 13

Neomysis 8 1 0 0 9

Sphaeroma 1 5 2 0 0

Flounder 1 7 1 1 0

Other fish 2 3 5 0 4

d 0.714 0.634 0.495 0.601 0.508

1 / d 1.41.4 1.58 2.022.02 1.67 1.96

Dominant species in flounder (Platichys flesus) diet across an Irish estuary (Wirjoatmodjo 1980)


Berger-Parker Calculations –Berger-Parker Calculations –Worked Example 11Worked Example 11

Calculate N, S, Nmax

Calculate d and 1/d Very simple



Week 1

Week 2


Alpha Diversity IndicesAlpha Diversity Indices “Heterogeneity Indices”

Consider both evenness AND richness Species abundance models only consider evenness

No assumptions made about species abundance distributions

Cause of distribution Shape of curve

“Non-parametric” Free of assumptions of normality


Two General CategoriesTwo General Categories Information Theory (complicated computation)

Diversity (or information) of a natural system is similar to info in a code or message

Examples: Shannon-Wiener and Brillouin Indices

Species Dominance Measures (simple comput.) Weighted towards abundance of the commonest

species Total species richness is downweighted relative to

evenness Examples: Simpson, McIntosh, and Berger-Parker


Information TheoryInformation Theory Information Theory, described (read more here)

A system contains more information when it has many possible states

• E.g., large numbers of species, or high species richness Also contains more information when the probability of

encountering each state is high • E.g., all species are equally abundant or have high evenness

Indices derived from this simple relationship between richness and evenness

Examples• Shannon-Wiener and Brillouin



Week 1

Week 2


Shannon-Wiener IndexShannon-Wiener Index Derived by Claude Shannon and Warren Weaver

in late 40s Developed a general model of communication and

information theory Initially developed to separate noise from information

carrying signals

Subsequently mathematician Norbert Wiener contributed to the model

as part of his work in developing cybernetic technology

Called alternatively Shannon-Weaver, Shannon-Wiener, or Shannon Index – more info here


Shannon-WienerShannon-Wiener Assumptions

All individuals are randomly sampled Population is indefinitely large, or effectively infinite All species in the community are represented

Result: difficult to justify for many communities Particularly very diverse communities, guilds, functional

groups Incomplete sampling significant error & bias

• Increasingly important as proportion of species sampled declines

• Simple mathematical consequence – see next slide


Shannon-Wiener Shannon-Wiener MathematicsMathematics

Equation H’ = -pi ln pi

• pi = proportion of individuals found in the ith species

• Unknowable, estimated using ni / N

– Flawed estimation, need more sophisticated equation (2.18 in Magurran)

Error • Mostly from inadequate sampling

• Flawed estimate of pi is negligible in most instances from this simple estimate


Shannon-Wiener Mathematics

Need to convert data Log2 was historically used Any Log base is acceptable

• Need consistency across samples Currently, Ln is used more commonly

• What we will use

Range of S-W index Usually between 1.5 and 3.5 Rarely surpasses 4.5 If underlying distribution is log-normal

• Need 100,000 species to have a H’ > 5.0


Building on H’ Can also use Exp H’

= Number of equally common species required to produce a given H’ value

• Reduces S from the observed value • Allows for an estimation of departures from maximal evenness

and diversity We won’t explore this here


Building on H’ Evenness measure (E)

Useful for determining the departure from maximal evenness and diversity

• Similar to the Exp H’ Hmax = maximal diversity which could occur if all species

collected were equally abundant E = H’ / Hmax = H’ / ln S 0 < E < 1

• H’ will always be less than Hmax Assumes all species have been sampled Some have criticized this as being biologically

unrealistic• Argue for best fit to the Broken Stick model


Comparing H’ Values Using Shannon for a t-test

Can use a simple t-test for differences between two samples

Need variance in H’ (Var H’) and to know the df• Both have complicated equations (2.19, 2.21 in Magurran)

Shannon and ANOVA H’ values tend to be normally distributed Can use ANOVAs for differences between multiple sites

• Need to have real replication to do this• Pseudoreplication introduces error, particularly in parametric

statistics


Shannon-Wiener Calculation – Worked Example 7

Calculate proportion of individuals in each species (pi) and ln pi

Sum all (pi)(ln pi) values

Calculate E E = H’ / ln S

Calculate Var H’ Var H’= ([pi)(ln pi)2 – pi)(ln pi))2] N) – ((S-1)/(2N2))

Calculate t t = (H’1 - H’2) / (Var H’1 + Var H’2)1/2

Calculate df df = (Var H’1 + Var H’2)2 / ([(Var H’1)2 / N1] + [(Var H’2)2 / N2])



Week 1

Week 2


Brillouin IndexBrillouin Index Useful when

The randomness of a sample is not guaranteed• Light traps, baited traps, attractive traps in general

Community is completely (thoroughly) censused• Similar to Shannon-Wiener index

Assumes Community is completely sampled Does not assume:

• Randomness of sampling• Equal attractiveness of traps


Brillouin MathematicsBrillouin Mathematics HB

Rarely larger than 4.5 Ranges between 1 and 4 most commonly


Brillouin vs. Shannon-WienerBrillouin vs. Shannon-Wiener Give similar values – significantly correlated Brillouin < Shannon-Wiener Brillouin has no uncertainty about all species present in

sample Does not estimate those that were not sampled, as in Shannon

When relative proportions of spp are consistent, totals differ

Shannon stays constant Brillouin will decrease with fewer total individuals

Brillouin is more sensitive to overall sample size Collections are compared, not samples

Disallows statistical comparisons, as all collections are different


Brillouin MathematicsBrillouin Mathematics Uses factorials throughout Equation

HB = (ln N! – ln ni!) / N

Evenness E = HB / HBmax

HBmax HBmax = [(1/n)][(ln {((N!) / (((N/S)!)s-r)*((((N/S)+1)!)r)}]

r r = N – S (N/S)


Brillouin Calculations – Brillouin Calculations – Worked Example 8Worked Example 8

Calculate HB Calculate r Calculate HBmax

Calculate E



Week 1

Week 2


Jack-Knifing Diversity IndicesJack-Knifing Diversity Indices Improves the accuracy of any estimate First proposed in 1956 (Quenouille) and refined by

Tukey in 1958 Theoretical biostatisticians First applied to diversity by Zahl in 1977


Jack-Knifing Jack-Knifing Assumptions:

None made about underlying distribution Does not attempt to estimate actual number of species

present• As in Shannon-Wiener

Random sampling is not necessary Repeated measures overcome the biases Jack-Knifing can determine the impact of biased

sampling


Data for Jack-KnifingData for Jack-Knifing Need multiple samples to conduct this procedure

Some debate exists about this, may be able to do a single sample

For our data Can use each tray to create an estimate for what? Can use each garden to create an estimate for what?


Jack-Knifing ProcedureJack-Knifing Procedure Procedure

Create the overall pooled index estimate ( Subsamples with replacement from the actual data Creates pseudovalues of the statistic Pseudovalues are normally distributed about the mean Mean value is best estimate of the statistics

Confidence limits Also possible to attach these to the estimate Consequence of normal distribution of pseudovalues


Applications of the Jack-KnifeApplications of the Jack-Knife Most commonly used for the most common

indices Shannon and Simpson in particular Also useful for other indices

Variance in the pseudovalues More useful than the Var H’ of the Shannon Gives a better estimate of the accuracy and impact of

non-random sampling


Jack-Knifing – Jack-Knifing – Worked Example 12Worked Example 12

Overall diversity index including all data (V) Recalculate, excluding each sample in turn

Creates n number of VJi estimates

Convert VJi to pseudovalues VPi Use VPi = (nV) – [(n-1) (VJi)] n = number of samples

Calculate mean VP value Calculate Sample Influence Function

SIF = V – VP

Calculate standard error VP = stand dev Vpis / n



Week 1

Week 2


Pielou’s Pooled Quadrat Pielou’s Pooled Quadrat MethodMethod

Similar to Jack-Knifing Improves the estimate of diversity

Also not influenced by non-random sampling Provides the best estimate of the value, given the data

Can be calculated using either of the information statistic indexes

Shannon Brillouin


Pielou’s Pooled QuadratPielou’s Pooled Quadrat Outputs

A graph that levels off when diversity has been best estimated in the community (Hpop)

Determine the minimal number of samples to achieve maximal diversity (t)

Quadrats

Diversity

t

Hpop

0

1

2

3


Pielou’s Method UtilityPielou’s Method Utility Stability of an index

Evaluating the stability of a diversity index and its relationship to sample size

Determining an adequate sample size Produces a graph of the indices When the line levels out, you have adequate samples

• Adequately estimated biodiversity locally

Can create confidence limits Then, can compare values between habitats Use standard parametric statistics


Pielou’s Pooled Quadrat – Pielou’s Pooled Quadrat – Worked Example 13Worked Example 13

Using Brillouin index, calculate all HBk From k = 0 k = z k = number of samples z = total samples Mk = total abundance in k number of samples

Estimate t t = Point at which HBk levels off

Calculate Hpop Using k+t number of samples Calculate mean Hpop

Calculate standard deviation


Next week(s):Next week(s): Continuing Alpha Diversity Indices Read

Magurran Ch 2, pages 32-45 Magurran Worked Examples 6-13

We will continue conducting alpha diversity analyses next week


Hypothetical Model CurvesHypothetical Model Curves

100

10

1

0.1

0.01

0.001

10 20 30 40

Geometric SeriesLog Series

Log-Normal Series

Broken Stick Model

Per Species

Abundance

Species Addition Sequence

Documents

Alpha Diversity Indices James A. Danoff-Burg Dept. Ecol., Evol., & Envir. Biol. Columbia University