Maximum Entropy,

Maximum Entropy Production

and their

Application to Physics and Biology

Roderick C. Dewar

Research School of Biological Sciences

The Australian National University

Summary of Lecture 1 …

The problem

to predict the behaviour of non-equilibrium systems with many degrees of freedom

The proposed solution

MaxEnt: a general information-theoretical algorithm for predicting reproducible behaviour under given constraints

Boltzmann

Gibbs

Shannon

Jaynes

Part 1: Maximum Entropy (MaxEnt) – an overview

Part 2: Applying MaxEnt to ecology

Part 3: Maximum Entropy Production (MEP)

Part 4: Applying MEP to physics & biology

Dewar & Porté (2008) J Theor Biol 251: 389-403

• The problem: explaining various ecological patterns

- biodiversity vs. resource supply (laboratory-scale)

- biodiversity vs. resource supply (continental-scale)

- the “species-energy power law”

- species relative abundances

- the “self-thinning power law”

• The solution: Maximum (Relative) Entropy

• Application to ecological communities

- modified Bose-Einstein distribution

- explanation of ecological patterns is not unique to ecology

Part 2: Applying MaxEnt to ecology

• The problem: explaining various ecological patterns

- biodiversity vs. resource supply (laboratory-scale)

- biodiversity vs. resource supply (continental-scale)

- the “species-energy power law”

- species relative abundances

- the “self-thinning power law”

• The solution: Maximum (Relative) Entropy

• Application to ecological communities

- modified Bose-Einstein distribution

- explanation of ecological patterns is not unique to ecology

Part 2: Applying MaxEnt to ecology

Ln (nutrient concentration)

unimodal

1. biodiversity vs. resource supply

bacteria

laboratory scale (Kassen et al 2000)

continental scale (104 km2) (O’Brien et al 1993)

monotonic

woody plants

Barthlott et al (1999)

Wright (1983) Oikos 41:496-506

2. Species-energy power law

62.0ES

angiosperms

24 islands world-wide

# species (S)

Total Evapotranspiration, E (km3 / yr)

3. Species relative abundances

cn

xns

n

)( 1c

Mean # species with population n

1xMany rare species

Few common species

n

xn for large n

(Fisher log-series)

Volkov et al (2005) Nature 438:658-661

)(nsn

n2log

6 tropical forests

nx

Enquist, Brown & West (1998) Nature 395:163-165

4. Self-thinning power law

3/4 Nm

Lots of small plants

A few large plants

Can these different ecological patterns (i.e. reproducible behaviours)

be explained by a single theory ?

• The problem: explaining various ecological patterns

- biodiversity vs. resource supply (laboratory-scale)

- biodiversity vs. resource supply (continental-scale)

- the “species-energy power law”

- species relative abundances

- the “self-thinning power law”

• The solution: Maximum (Relative) Entropy

• Application to ecological communities

- modified Bose-Einstein distribution

- explanation of ecological patterns is not unique to ecology

Part 2: Applying MaxEnt to ecology

C is all we need to predict reproducible behaviour

Constraints C (e.g. energy

input, space)

Reproducible behaviour

(e.g. species abundance distribution)

Predicting reproducible behaviour ….

System with many degrees of

freedom (e.g. ecosystem)

pi = probability that system is in microstate i Macroscopic prediction:

Incorporate into pi only the information C

i

iiQpQ

MaxEnt

… more generally we use Maximum Relative Entropy (MaxREnt) …

i i

ii q

ppqpH log

qpH = information gained about i when using pi instead of qi

qi = distribution describing total ignorance about i

qpHMaximize w.r.t. pi subject to constraints C

pi contains only the information C

qi

pi

i i

ii q

ppqpH log

= information gained about i when using pi instead of qi

total ignorance about i

contains only the info. C

… ensures baseline info = total ignorance

Minimize:

Constraints C

• The problem: explaining various ecological patterns

- biodiversity vs. resource supply (laboratory-scale)

- biodiversity vs. resource supply (continental-scale)

- the “species-energy power law”

- species relative abundances

- the “self-thinning power law”

• The solution: Maximum (Relative) Entropy

• Application to ecological communities

- modified Bose-Einstein distribution

- explanation of ecological patterns is not unique to ecology

Part 2: Applying MaxEnt to ecology

r1

r2

rS

j = species labelrj = per capita resource use nj = population

n1

n2

nS

S

jjjrnR

1

S

jjnN

1

subject to constraints (C)

Maximize

q

ppqpH

jnlog

0

Application to ecological communities

p(n1…nS) = ?

where (Rissanen 1983)

S

j jS nnnq

11 1

1 ...microstate

The ignorance prior

S

j jS nnnq

11 1

1 ...

dxxqxdxq λλxxx λ

xqxq λλ

xqxq λλ x

xq1

For a continuous variable x (0,), total ignorance no scale

Under a change of scale …

… we are just as ignorant as before (q is invariant)

the Jeffreys prior

Solution by Lagrange multipliers (tutorial exercise)

where

modified Bose-Einstein

distribution

mean abundance of species j:

mean number of species with abundance n:

probability that species j has abundance n:

B-E

Example 1: N-limited grassland community (Harpole & Tilman 2006)

2m 62 N

S = 26 species (j = 1 …. 26)

1-2 yrm N g 5.9 R

rj

1-2 yrm N g 5.9 R

+2+4

+6

+81-2 yrm N g 5.9 R

rj (N use per plant)

Community nitrogen use, (g N m-2 yr-1)R

9)obs.(opt R

3.9)pred.(opt R

Predicted relative abundances

Shannon diversity index exp(Hn)

Example 2: Allometric scaling model for rj

Demetrius (2006) : α = 2/3

αmr

West et al. (1997) : α = 3/4

per capita resource use

adult mass

metabolic scaling exponent

α1 jrj

Let’s distinguish species according to their adult mass per individual

variable N

S =

α = 2/3

On longer timescales, S = and

variable N

S* = # species with 1jn 0μ

α1/1 * RS

MaxREnt predicts a monotonic species-energy power law

α1/1 * RS

60.0α1/13/2α

62.0ES

Wright (1983) :

57.0α1/14/3α

mean # species with population n vs. log2n

nxn

nnsn

1)(

)(ns

3/4 Nm

For large, is partitioned equally among the different species

Crrn cf. Energy Equipartition of a classical gas

r

rn1

αmr

α/1 nm

3/4α/14/3α

R R

Summary of Lecture 2 …

Boltzmann

Gibbs

Shannon

Jaynes

ecological patterns = maximum entropy behaviour

the explanation of ecological patterns is not unique to ecology