OVERVIEW L2. Model parameterization and validation

L2. Model parameterization and validation

OVERVIEW

. L1. Approaches to ecological modelling . L2. Model parameterization and validation

. L3. Stochastic models of population dynamics (math) . L4. Animal movement (math + stat)

. L5. Quantitative population genetics (math + stat) . L6. Community ecology (stat)

Data (e.g. time-series of population dynamics) Parameter

values

Model structure Knowledge about the

ecological context (e.g. life-history, …)

Model prediction

The forward approach to model parameterization

Measurements of model parameters

(e.g. life-span, …)

Level 1 (e.g. individual level) Level 2 (e.g. population level)

No

Model compatible with data. New data needed to refine the model or

to select between alternative models.

Yes

Model not compatible with data. Refine the

model.

Good match?

New data from a different context

Model structure

Posterior knowledge about parameter

estimates

The Bayesian state-space approach to model parameterization

Any kind of data that can inform the model parameters directly or indirectly

No

Model compatible with the data used to parameterize the model. New data needed to refine the model or

select between alternative models.

Yes

Good match?

Model prediction Prior knowledge

about ecological context

Prior knowledge about parameter values

Model not compatible with the data used to parameterize the model. Refine

the structural model assumptions.

Posterior knowledge about ecological context

Model prediction of

new data

Good match?

Yes Model and parameter may be of general nature

Model or parameters are of specific nature

No

State-space models

process model:

what happens in nature?

observation model:

how did we collect the data?

data

How to combine the forward and inverse-approaches in practice?

Also called Hidden Markov models, process based models...

Bayesian state-space models

prior:

what did we know about the process model parameters before collecting the data?

observation model:

how did we collect the data?

data process model:

what happens in nature?

prior:

what did we know about the observation model parameters

before collecting the data?

How to combine the forward and inverse-approaches?

Example: stochastic logistic model

Number of individuals is 𝑛 = 𝑛 𝑡 = 0,1,2,3, …

Individuals produce new individuals at per-capita fecundity rate 𝑓

The per-capita death rate is 𝑑 + 𝑐(𝑛 − 1), where 𝑑 is the density-independent background mortality rate and the parameter 𝑐 describes the additional death rate imposed by competitive effects of the 𝑛 − 1 individuals to the focal individual

𝑑𝑛

𝑑𝑡 = 𝑓 − 𝑑 𝑛 − 𝑐𝑛² = 𝑟𝑛(1 − 𝑛/𝐾),

The model is a stochastic Markov process. The deterministic mean-field model is

𝑟 = 𝑓 − 𝑑 is the growth rate of the population at low density 𝐾 = 𝑟/𝑐 is the carrying capacity

# individuals n

time t

Model simulation with “true” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Initial state 𝑛 0 = 5.

0 2 4 6 8 10

20 40 60 80

# individuals n

time t

Model simulation with “true” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Initial state 𝑛 0 = 5.

0 2 4 6 8 10

20 40 60 80

# individuals n

time t

Model simulation with “true” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Initial state 𝑛 0 = 5.

2 4 6 8 10

20 40 60

80 The data: 𝑦 = (20,40,43,40,53,64,48,50,41,42)

# individuals n

time t

“True” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Simulations with 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

0 2 4 6 8 10

20 40 60 80

# individuals n

time t

“True” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Simulations with 𝑓 = 2, 𝑑 = 1 and 𝐾 = 50.

0 2 4 6 8 10

20 40 60 80

# individuals n

time t

“True” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Simulations with 𝑓 = 1.5, 𝑑 = 1 and 𝐾 = 50.

0 2 4 6 8 10

20 40 60 80

# individuals n

time t

“True” parameter values 𝑓 = 3, 𝑑 = 1 and 𝐾 = 50.

Simulations with 𝑓 = 1, 𝑑 = 1 and 𝐾 = 50.

0 2 4 6 8 10

20 40 60 80

𝜃 = (𝑓, 𝑑, 𝐾)

𝑦 = (20,40,43,40,53,64,48,50,41,42)

Likelihood of observing the data

The data:

The parameters:

𝜃^TRUE = (3,1,50)

The probability (likelihood) of observing the data,

given the model and the model parameters:

𝑛 1 |𝑛 0 = 5

Fraction out of 100,000 simulations

𝑦 = (20,40,43,40,53,64,48,50,41,42)

How to compute the likelihood of observing the data

The data:

The experiment was initiated at day 0 with 5 individuals. What is the

probability that there would be 20 individuals at day 1, assuming 𝜃

?

𝑝₁ = 𝑃 𝑛 1 = 20 𝑛 0 = 5 = 0.03815 𝑝₂ = 𝑃 𝑛 2 = 40 𝑛 1 = 20 = 0.043095

𝑝₁₀… = 𝑃 𝑛 10 = 42 𝑛 9 = 41 = 0.0368283

𝑝 𝑦 𝜃^TRUE = 𝑝₁𝑝₂… 𝑝₁₀

= 0.00000000000000197577

= 1.97577 ∙ 10⁻¹⁵

0 2 4 6 8 0.5

1.0 1.5 2.0 2.5 3.0

How does the likelihood depend on model parameters?

𝜃^TRUE = 𝑓, 𝑑, 𝐾 = (3,1,50)

Keep 𝑑, 𝐾 = (1,50) and vary f

fecundity f

lik elihood of th e da ta 𝑝( 𝑦| 𝜃 )

× 10⁻¹⁵

0 2 4 6 8 0.5

1.0 1.5 2.0 2.5 3.0

How does the likelihood depend on model parameters?

𝜃^TRUE = 𝑓, 𝑑, 𝐾 = (3,1,50)

Keep 𝑑, 𝐾 = (1,50) and vary f

fecundity f

Keep 𝑑, 𝐾 = (0.5,50) and vary f

Pr ob ability of the da ta 𝑝( 𝑦| 𝜃 )

× 10⁻¹⁵

Bayesian inference

𝑝 𝑦 𝜃 : the likelihood of observing the data 𝑦 conditional on the parameters 𝜃 𝑝(𝜃|𝑦): the posterior: the probability distribution of parameters, given 𝑦.

𝑝 𝜃 : the prior: what we assumed about the parameters before seeing the data

Bayes theorem:

𝑝 𝜃 𝑦 ∝ 𝑝 𝜃 𝑝 𝑦 𝜃

How to choose the prior

Sometimes there is prior information, e.g. from other studies.

“We followed singly grown individuals through their life-times, from which data we estimate the density-independent death rate 𝑑 to be between 0.7 and 1.1.”

Often there is no prior information. Then one may assume an “uninformative prior”.

Let us assume, for the sake of illustration, the following prior:

𝑝 𝜃 = 1/20000 if 0 ≤ 𝑓, 𝑑 ≤ 10 and 0 ≤ 𝐾 ≤ 200

otherwise 𝑝 𝜃 = 0

The marginal posterior distributions

O u t[31 0 ]=

𝜃^TRUE = 𝑓, 𝑑, 𝐾 = (3,1,50)

probability density

fecundity f mortality d carrying capacity K

48.1 (41.6-58.6) 1.42 (0.06-5.88)

3.66 (1.48-9.22)

posterior mean (95% credibility interval)

The joint posterior distribution

fecundity f

mortality d

0 2 4 6 8 10

0 2 4 6 8 10

𝜃^TRUE = 𝑓, 𝑑, 𝐾 = (3,1,50)

The posterior distribution of a derived parameter

probability density

growth rate 𝑟 = 𝑓 − 𝑑 2.2 (0.81-4.81)

𝜃^TRUE = 𝑓, 𝑑, 𝐾 = (3,1,50)

Individual-level

model observation model

data

prior prior

Population-level model

prior

State-space models often have a hierarchical structure

Metapopulation Research Group

50 by 70 km

Example: Glanville fritillary metapopulation dynamics

prof. Ilkka Hanski

Building metapopulation dynamics from individual behavior

,

Ni t

, u

Ni t

, u

Si t

,

Si t

,

Fi t

et

e_ e_



, o

Si t , o

Ni t ,

pi t

0

nh ,

j t j ì

F _{ }

,

Mi t

,

Gi t

,

Qi t

,

Ei t t_t^

t_^ t_^

ji j ì

R _{ }

T

Ti

ki

,1

pMR

1 2

D D m k_ MR

, 2

pMR

p_

p_

, , c y s

p

,

Nc y , ,

o c y s

N pair s

year y patch c

patch i

year t

i

Ki

q_ qr

qe

qk

dist rad

radi

over-winter survival

autumn & spring nest data

eclosing of adult females

movement &

oviposition

nest spinning control

monitoring data

mark-recapture data

Harrison, P. J., Hanski, I. and Ovaskainen, O. 2012. Bayesian state-space modeling of metapopulation dynamics in the Glanville fritillary butterfly. Ecological Monographs 81, 581-598.

Part of the individual-based model

. The number of larval groups that

survive through the winter: N ~ Bin E( , )



~ ( )

E Poisson



The number of egg groups in the autumn:

 _{E N}

The amount of time all females spend in a patch

1 2

3 4 5

6 7

8

9 10

11

13 12 14

15 17 16

18

19 20

2122 23 24

25

26 27 28

29 30

31

32 33

34

35 36

38 37

39 40

41 42

43

44

45

46 47

48 49

3.110 106 3.111 106 3.112 106 3.113 106 3.114 106

6.712 106 6.713 106 6.714 106 6.715 106 6.716 106

LOCATION TIME EGG GROUPS

Patch 23 1.92444 0

Matrix 0.267237 0

Patch 45 1.19267 1

Matrix 0.607294 0

Patch 8 0.124928 0

Life of a single female

5 km

Example of model prediction at the individual level

Circle = habitat patch

5 km

Number of larval groups

Circle = habitat patch

Examples of model prediction at the population level

70 km

Fraction of occupied patches

Circle = habitat patch network

Examples of model prediction at the metapopulation level

Strategies for model validation

1. Do nothing, just trust the model (still most common option!)

2. Fit model to data, then check if the model can reproduce the same data

3. Cross-validation: split the data into two parts. Use data 1 for fitting the model, and check if the model is able to reproduce data 2

4. If you have data from different situations, see if the

model fitted to situation 1 can reproduce the data

collected from situation 2

0 500 1000 0

1 10 100

prediction

Distance moved (m)

Frequency

Example of model validation (strategy 4)

Model parameterized with data from Landscape A, prediction for Landscape B

Empirical data on Landscape B

Landscape A Landscape B

Ovaskainen, O., Luoto, M., Ikonen, I., Rekola, H., Meyke, E. and Kuussaari, M. 2008. An empirical test of a diffusion model: predicting clouded apollo movements in a novel environment. American Naturalist 171, 610-619.

Strategies for model selection

1. Try only one model and hope it fits nicely enough (still most common option!)

2. See which model reproduces the data (or preferably some independent data) using a summary statistic you best like / think is biologically relevant

3. Use formal model selection methods: AIC, BIC, DIC,

Bayes factor, ...

L2: take home messages

• State-space models combine a process model with an observation model. They provide a very general framework of formulating and fitting movement models.

• State-space models can be visualized using a DAG (directed acyclic graph).

DAG is a very useful way to illustrate how the components of the model link to each other.

• State-space models allow one to bring biological knowledge into statistical inference, parameterize dynamic models of movement, and to use data with missing observations.

• Fitting state-space models to data can be technically challenging. A great number of methods exist (essentially variants of MCMC approaches).

• All models are wrong, but some are still useful. Take model selection and validation seriously!