Semi-mechanistic modelling in Nonlinear Regression: a case study

by Katarina Domijan1, Murray Jorgensen2 and Jeff Reid3

1AgResearch Ruakura 2University of Waikato

3Crop and Food Research

Introduction

Reid (2002) developed a crop response model

genetic algorithm

no measures of confidence for individual parameter estimates

General structure

non-linear model semi-mechanistic

parts of the model that would ideally be mechanistic are replaced by empirically estimated functions

relatively complex (26 parameters to be estimated)

generality of application

challenging to fit

General structure

water stress,plant density,

quantity of lightetc

yield under ideal nutrient/pH conditions

(maximum yield)nutrient supply

(N, P, K, Mg)

observed yield

These effects are assumed to act multiplicatively and

independently of each other and the nutrient effects

General structure – maize data

Crop hybrid

0 110 220 330Nsupply

0.6

1.1

sca

led

ob

serv

ed

yie

ld

347634933522373037533905

300 800 1300Ksupply

0.6

1.1

sca

led

ob

serv

ed

yie

ld

347634933522373037533905

30 80 130Psupply

0.6

1.1

sca

led

ob

se

rve

d y

ield

347634933522373037533905

300 800 1300

Mgsupply

0.6

1.1

sca

led

ob

se

rve

d y

ield 3476

34933522373037533905

300 800 1300

Mgsupply

0.6

1.1

scale

d o

bse

rved

yield

347634933522373037533905

300 800 1300

Mgsupply

0.6

1.1

scale

d o

bse

rved

yield

347634933522373037533905

General structure - nutrients

Model structure is the same for all nutrients

Each nutrient supply is assumed to have:a minimum value (the crop yield is zero) and an optimal value (further increases cause no additional

yield)

Reid (2002) defines a scaled nutrient supply index which is: = 0 at the minimum nutrient value and = 1 at the optimal nutrient value

In soiladded as fertilizer

nutrient supply

efficiency factor1

efficiency factor2

Proportion of the optimum amount of nutrient supply

General structure - nutrients

0.0 0.5 1.0 1.5 2.0

0.0

0.4

0.8

1.2

nutrient supply index

sca

led

yie

ld

γ1+γ γxγ)x(1g(x) yieldscaled γ1+γ γxγ)x(1g(x) yieldscaled

For each nutrient, the effect of scaled nutrient supply index (x) on yield is modelled using the family of curves:

Nopt

where:γ =shape parameter

0 g(0)

General structure - combining nutrients

The combined scaled yield is given by:

(or 0 if this is negative).

Note: are scaled yields corrected for unavailability of the respective nutrients Nutrient stresses are assumed to affect yield independently of each other

Soil pH Treated as if it were an extra nutrient Only stress due to low pH is modelled and not stress due to excessive pH

2 4 3

2 2

2 1 )) g(x (1 )) g(x (1 )) g(x (1 )) g(x 1 ( 1

2

if the effects of a particular stressor are known to be

absent for a set of data, then

) g(x , ) g(x

), g(x ), g(x

4 3 2 1

1 g(1)

just 2 nutrients (N and K)

Scaled yield

General structure - combining nutrients

Data

model was tested for maize crops grown in the North Island between 1996 and 1999

data was collated from 3 different sources of measurements

experimental and commercial crops 12 sites 6 hybrids 84 observations

Genetic Algorithms

stochastic optimization tools that work on “Darwinian” models of population biology

don’t need requirement of differentiability! relatively robust to local minima/maxima don’t need initial values

have no indication of how well the algorithm has performed

convergence to a global optimum in a fixed number of generations?

slow to move from an arbitrary point in the neighbourhood of the global optimum to the optimum point itself

no measure of confidence for individual parameters

Our approaches:

simplifying the model: 1 nutrient (N) 9 parameters

simulated data combining GA with derivative based methods:

common methods (Gauss-Newton, Levenberg-Marquardt)

AD MODEL BUILDER obtain CI’s:

gradient information likelihood methods

Correlation of Parameter Estimates:

g Nm Np d b e1 e2 E.1Nmin B 1 Nopt . . 1 delta 1 beta , 1 eta1 . 1 eta2 1 E.n1 , 1 E.n2 , +

Simulated data - simple model

investigate the structure of the correlation matrix generated so it mimics the “real” data as much as possible large n (300), small residual variance (0.01)

Nmin and γN are highly correlated!

(blank) 0-0.3

. 0.3-0.6

, 0.6-0.8

+ 0.8-0.9

* 0.9-0.95

B 0.95-1

Key:

Correlation of Parameter Estimates:Nm No gN E.n1 E.n2 Pm Po gP E.p1 E.p2 Km Ko gK E.k1 E.k2 Mm Mo gM E.M1 E.M2 b D e1 e2 pH.cNmin 1Nopt , 1 gammaN + , 1 E.n1 . , 1 E.n2 . , + 1 Pmin 1 Popt , 1 gammaP , * 1 E.p1 B , , 1 E.p2 * . . B 1 Kmin 1 Kopt 1 gammaK B + 1 E.k1 1 E.k2 , 1 Mmin 1 Mopt . 1 gammaM , + 1 E.M1 1E.M2 1 beta 1 delta , 1 eta1 1 eta2 . 1 pHcrit 1 lambda.pH +

Complete model

n=50000, σ2=0.0001

Key:

(blank) 0-0.3

. 0.3-0.6

, 0.6-0.8

+ 0.8-0.9

* 0.9-0.95B 0.95-1

130

140

150

160

-20 -15 -10 -5 0

0.5

1.0

1.5

2.0

2.5

3.0

Nmin

N

Nmin

Levenberg-Marquardt algorithm

Maize data use GA estimates as starting values simple model:

multicollinearity parameter Nmin tends to –ve reparametrization

complete model: (again) biological restrictions (Nmin, Kmin=0) problems with equations which are constant for ranges of values

(eg scaled yields) replace nondiffentiable functions (pH, water stress) some stressors held constant (P, Mg)

0.0 0.5 1.0 1.5 2.0

0.0

0.4

0.8

1.2

nutrient supply index

sca

led

yie

ld

Nopt

constant

Levenberg-Marquardt algorithm

Levenberg-Marquardt algorithm

2 nutrients (N and K) + stress due to low pH + water and population stresses

12 parameters

20 40 60 80 120

-2-1

01

23

plot of residuals on Psupply

Psupply

e

200 600 1000

-2-1

01

23

plot of residuals on Mgsupply

Mgsupply

e

Profile likelihood CI’s

5 10 15 20 25 30

-4

-3

-2

-1

0

1

2

Nopt

tau

0.0 0.2 0.4 0.6 0.8 1.0 1.2

-4

-3

-2

-1

0

1

2

gammaN

tau

0.0 0.5 1.0 1.5

-4

-3

-2

-1

0

1

2

E.n1

tau

40 60 80 100 120

-2

-1

0

1

2

Kopt

tau

0.1 0.2 0.3 0.4

-4

-3

-2

-1

0

1

2

gammaK

tau

-4 -2 0 2 4 6

-4

-3

-2

-1

0

1

2

E.k1

tau

5 10 15 20 25 30

-4

-3

-2

-1

0

1

2

Nopt

tau

5 10 15 20 25 30

-4

-3

-2

-1

0

1

2

Nopt

tau

0.0 0.2 0.4 0.6 0.8 1.0 1.2

-4

-3

-2

-1

0

1

2

gammaN

tau

0.0 0.2 0.4 0.6 0.8 1.0 1.2

-4

-3

-2

-1

0

1

2

gammaN

tau

0.0 0.5 1.0 1.5

-4

-3

-2

-1

0

1

2

E.n1

tau

0.0 0.5 1.0 1.5

-4

-3

-2

-1

0

1

2

E.n1

tau

40 60 80 100 120

-2

-1

0

1

2

Kopt

tau

40 60 80 100 120

-2

-1

0

1

2

Kopt

tau

0.1 0.2 0.3 0.4

-4

-3

-2

-1

0

1

2

gammaK

tau

0.1 0.2 0.3 0.4

-4

-3

-2

-1

0

1

2

gammaK

tau

-4 -2 0 2 4 6-4 -2 0 2 4 6

-4

-3

-2

-1

0

1

2

E.k1

tau

-4

-3

-2

-1

0

1

2

E.k1

tau

Nopt

estimate

Wald CI

Likelihood CI

Approach outlined in Bates and Watts (1989)

Assess validity of the linear approximation to the expectation surface

Profile likelihood CI’s

Estimation surface seems to be nonlinear with respect to most of the parameters in the model

Especially EN1 and pHc -> one sided CIs

Better estimates of uncertainty than linear approx. results

-1.0 -0.5 0.0 0.5 1.0

-3

-2

-1

0

1

2

EK2

tau

2 4 6 8 10

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

pHc

tau

0.6 0.8 1.0 1.2

-3

-2

-1

0

1

2

3

β

tau

0.2 0.3 0.4 0.5 0.6 0.7

-3

-2

-1

0

1

2

3

δ

tau

0.2 0.3 0.4 0.5 0.6 0.7

-3

-2

-1

0

1

2

3

η1

tau

0.5 0.6 0.7 0.8

-3

-2

-1

0

1

2

3

η2

tau

-1.0 -0.5 0.0 0.5 1.0

-3

-2

-1

0

1

2

EK2

tau

-1.0 -0.5 0.0 0.5 1.0

-3

-2

-1

0

1

2

EK2

tau

2 4 6 8 10

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

pHc

tau

2 4 6 8 10

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

pHc

tau

0.6 0.8 1.0 1.2

-3

-2

-1

0

1

2

3

β

tau

0.6 0.8 1.0 1.2

-3

-2

-1

0

1

2

3

β

tau

0.2 0.3 0.4 0.5 0.6 0.7

-3

-2

-1

0

1

2

3

δ

tau

0.2 0.3 0.4 0.5 0.6 0.7

-3

-2

-1

0

1

2

3

δ

tau

0.2 0.3 0.4 0.5 0.6 0.7

-3

-2

-1

0

1

2

3

η1

tau

0.2 0.3 0.4 0.5 0.6 0.7

-3

-2

-1

0

1

2

3

η1

tau

0.5 0.6 0.7 0.8

-3

-2

-1

0

1

2

3

η2

tau

0.5 0.6 0.7 0.8

-3

-2

-1

0

1

2

3

η2

tau

5 10 15 20 25 30

-4

-3

-2

-1

0

1

2

Nopt

tau

0.0 0.2 0.4 0.6 0.8 1.0 1.2

-4

-3

-2

-1

0

1

2

γN

tau

0.0 0.5 1.0 1.5

-4

-3

-2

-1

0

1

2

EN1

tau

40 60 80 100 120

-2

-1

0

1

2

Kopt

tau

0.1 0.2 0.3 0.4

-4

-3

-2

-1

0

1

2

γK

tau

-4 -2 0 2 4 6

-4

-3

-2

-1

0

1

2

EK1

tau

5 10 15 20 25 30

-4

-3

-2

-1

0

1

2

Nopt

tau

5 10 15 20 25 30

-4

-3

-2

-1

0

1

2

Nopt

tau

0.0 0.2 0.4 0.6 0.8 1.0 1.2

-4

-3

-2

-1

0

1

2

γN

tau

0.0 0.2 0.4 0.6 0.8 1.0 1.2

-4

-3

-2

-1

0

1

2

γN

tau

0.0 0.5 1.0 1.5

-4

-3

-2

-1

0

1

2

EN1

tau

0.0 0.5 1.0 1.5

-4

-3

-2

-1

0

1

2

EN1

tau

40 60 80 100 120

-2

-1

0

1

2

Kopt

tau

40 60 80 100 120

-2

-1

0

1

2

Kopt

tau

0.1 0.2 0.3 0.4

-4

-3

-2

-1

0

1

2

γK

tau

0.1 0.2 0.3 0.4

-4

-3

-2

-1

0

1

2

γK

tau

-4 -2 0 2 4 6-4 -2 0 2 4 6

-4

-3

-2

-1

0

1

2

EK1

tau

-4

-3

-2

-1

0

1

2

EK1

tau

AD Model builder

automatic differentiation faster observed information matrix (better se’s)

we run into the same problems as with L-M requires model to be differentiable good initial values

In the end...

CI’s are too wide to be of ‘practical’ use e.g. for parameter Nopt (optimum amt of N supply per tonne of

maximum yield) :

but in the ‘maize dataset’ N supply per tonne of maximum yield varies between 6 and 54

Problems of nonidentifiability correlated estimates poor precision of estimation in certain directions

These phenomena are not clearly distinguished in nonlinear setting

L-M and ADMB estimate

95% LINEAR APPROX. CI’s

95% PROFILE LIKELIHOOD CI’s

95% CI’s (ADMB)

18.575 (3.80, 33.35) (11.67, 31.45) (11.32, 25.83)

Recommendations

do more experimentation - collect more information about parameters

particularly ‘approximately nonidentifiable’ parameters

replace all nondifferentiable equations in the model with smooth versions

bootstrapping

global optimum?