A Latent Approach to the Statistical Analysis of Space-time Data

A Latent Approach to the Statistical Analysis of Space-time Data

Dani Gamerman

Instituto de Matemática

Universidade Federal do Rio de Janeiro Brasil

http://acd.ufrj.br/~dani

17th International Workshop on Statistical Modelling Chania, Crete, Greece, 8-12 July 2002

1990 1974

Played in Europe

World Cup Algorithm

?

2006 1958

Played in Europe

2002 1962

Played in exotic place

1970

1994

Played in (L) America

1986 1978

Played in L. America

1966

1998

 One-time only home win Europe  25 miles apart

Center of the world football

1982

A Latent Approach to the Statistical Analysis of Space-time Data

Dani Gamerman

Instituto de Matemática

Universidade Federal do Rio de Janeiro Brasil

http://acd.ufrj.br/~dani

Joint work with

Marina S. Paez (IM-UFRJ) Flavia Landim (IM-UFRJ) Victor de Oliveira (Arkansas)

Alan Gelfand (Connecticut) Sudipto Banerjee (Minnesota)

17th International Workshop on Statistical Modelling

Introduction

Examples:

1) measurements of pollutants in time over a set of monitoring stations

3) counts of morbidity/mortality events in time over a collection of geographic regions

Environmental science – data in the form

of a collection of time series that are geographically referenced.

Some examples can be found in other areas

2) selling price of properties around a neighborhood of interest

Main Objective: spatial interpolation

Example: Pollution in Rio de Janeiro

Paez, M.S. and Gamerman, D. (2001). Technical report. Statistical Laboratory, UFRJ.

Example: Pollution in Rio de Janeiro Prob ( PM10 > 100 g/m³ | Y^obs )

Other features of interest can be obtained Picture showed mean interpolated values

Spatial Interpolation

m = number of observations g = number of grid points s₁, ... ,s_m = observed sites

s₁ⁿ,...,s_gⁿ = grid points (to interpolate)

Y₁ⁿ,...,Y_gⁿ = observations in the grid points

 ^{  }

 p Y Y p Y d Y

Y

p (

|

) (

|

, ) ( |

)

 - all model parameters

Y^mis - missing data, treated as parameters

1. Frequentist inference: generate Yⁿ from

p ( Y

| Y

,   )

• Obtain P(Yⁿ|Y^obs)by simulation.

Steps to generate from Yⁿ|Y^obs : If  with probability 1 then

) ,

| (

)

|

( Y

Y

 p Y

Y



p

2. Bayesian inference: i ) generate  from ii ) generate Yⁿ from

)

|

( Y

p 

) ,

|

( Y

Y

 p

Interpolation p ( Y

| Y

)

Usual simplifications:

where

 = ( ₁, ... , _n ) with _i=E[w(s_i)] and

 = (_i_j [w(s_i), w(s_j)] )_i,j

Gaussian Process (GP)

(or Gaussian Random Field)

S - region of R^p (in general, p=2) { w(s) : s S } is a GP if

n, s₁ , ... , s_m  S

( w(s₁) , ... , w(s_n) ) ~ N_n (, )

2) Homoscedasticity _i = , i Notation: w(.) ~ GP((.),,_)

1) Isotropy [w(s_i),w(s_j)]= _(h_ij) with h_ij=|s_i– s_j|

Statistical Analysis

Starting point: regression models Y_t(s) = _t(s) + e _t(s) where

_t(s) = ₀ + ₁ X _t1(s) + ... + _pX_tp(s) and e_t(s) ~ N(0, _e²) independent

Suppose that X_tj(s) handles temporal autocorrelation Otherwise, we can include a temporal component _t

Usually e_t(s) remains spatially correlated In this case, e_t(s) = e₀(s) + e_t1(s)

e₀(s)  errors spatially correlated e_t1(s)  pure residual (white noise)

 ₀(s) = ₀ + e₀(s)

Inference

1. At first (3 steps)

• How to estimate ₀(s) ?

Traditional approach: geostatistical ₀(.) ~ GP(₀,,_) or

e₀(.) = ₀(.)  ₀ ~ GP(0,,_)

(b) _e², ² and ₀ estimated from r_t0(s)

ˆ0

ˆ and

ˆ , 

_e

(c) Inference based on

(a) ₀ , ₁ , ... , _p estimated in the regression model and the residuals r_t0(s) = Y_t (s)  _t(s) are constructed

then, ₀^obs ~ N(₀ 1, , R)

₀^obs = (₀(s₁) , ... , ₀(s_m) )

Hiperparameters: _e², ² and ₀

3) Natural solution (Kitanidis, 1986; Handcock & Stein, 1993):

• specify distribution for ₀

• perform Bayesian inference Problems:

(a) r_t0(s)  e_t (s)

(b) ^( ^{ˆ e,ˆ,ˆ0) (e,,0)}

2) next step:

• ₀ , ₁ , ... , _p and estimated jointly  solves (a)

• but to incorporate uncertainty about is complicated

, 0

, 

_e

ˆ0

ˆ, ˆ , 

_e

Recall: E[Y_t(s)]=₀(s) + ₁X_t1(s) + ... + _pX_tp(s)

Spatial heterogeneity doesn’t have to be restricted to ₀ Model generalization

Example:

site by site effect of temperature in the Rio pollution data

Extension of the previous model

E [Y_t(s)] = ₀(s) + ₁(s)X_t1(s) + ... + _p(s)X_tp(s) previous model

E [Y_t(s)] = ₀(s) + ₁ X_t1(s) + ... + _p X_tp(s)

Hyperparameters: = (_e_where  = (₀, ₁,..., _p) Special cases for the _j(.)´s:

One possibility: (.) ~ GP(, , _)

a) prior independence

) ,...,

(

diag  





(.)) (.),...,

( 







b) same spatial structure and prior correlation between the _j(.)´s

j

   j

 (.) (.),

We can accommodate spatial variation for other coefficients _j, j=1, ... , p.

(.) = (₀(.), ₁(.),..., _p(.))

How to estimate _j(s), j=0,1,...,p ?

2) natural solutions:

Specify prior distribution for 

In general, independent and non informative priors are used Problems (the same as before):

(a) b_j(s)  _j(s)

(b) ^(ˆ,ˆ ^{e,ˆ,ˆ) (,e,,)}

1) classical solution (Oehlert, 1993; Solna & Switzer, 1996):

(a) ₀ (s), ₁ (s), ... , _p (s) estimated by

b₀(s), b₁(s), ... , b_p (s) in the local regression model (b) estimated from the b_j(s)

(c) inference based on ,_e,, ˆ,ˆ _e,ˆ,ˆ

Model Summary

Parameters: ^obs , where  = ( ,_e², ,)

_j^obs = (_j(s₁) , ... , _j(s_m) ), j=0, 1, ... , p

^obs = (₀^obs , ... , _p^obs )

= ( ₀ , ₁ , ... , _p )

Data: Y^obs = (Y₁(s₁) , ... , Y_T(s_m)) X^obs = (X₁(s₁) , ... , X_T(s_m))

Simulated data

Y_t(s) = _t(s) _t(s), t=1,...,30 _t(s) = ₀(s)+ ₁(s) X_t(s)

_t(s) ~ N(0, _e²) independent with _e²=1

0 ~ N(_, _,_(

1 ~ N(_, _,_(

X_t(s) _~ N(_, _,_(, for all time t

Exponential correlation functions: _j(x)exp{- _j x}

_{0}= 100 _{1}= 5 _2= 0

_0= 0.4 _1= 0.8 _2= 1.5 ₀²_= 0.1 ₁²_= 1 ₂²_= 0.333

+

=

+

₀

₁X



0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

-2.4 -2.4

-2.4

-1.7

-1.7

-1.7 -1.7

-1.7

-1.7 -0.9

-0.9 -0.9

-0.9

-0.9 -0.9

-0.9

-0.2

-0.2 -0.2

-0.2

-0.2

-0.2 -0.2 -0.2

-0.2

-0.2 -0.2

-0.2 -0.2

0.5 0.5

0.5

0.5 0.5

0.5

0.5 0.5

0.5

0.5 0.5

0.5

1.3 1.3

1.3

1.3 2.0

2.0

2.0 2.0

2.7 2.7

Y

Simulated Data

Inference

Parameters: (^obs ,)

= ( ,_e², ,) Likelihood:

L(^obs ,) = p(Y^obs | ^obs , _e² ) Prior:

p(^obs ,)= p( ^obs | ) p() p(_e²) p() p() Posterior:

(^obs ,)  L (^obs ,)  p(^obs ,)

• Many parameters

• Complicated functional form

• Solution by MCMC

 again, use _j^obs as data (geostatistical analysis) (c) [ _e² | rest ] ~ [ _e² | Y^obs , ^obs ]

~ Inverse Gamma

Full Conditionals

(a) [ ^obs | rest ] ~ Normal (b) [  | rest] ~ Normal

(e) [  | rest ] ~ _j p(_j | _j^obs , _j, ) p()

 use _j^obs as if they were data

 hard to sample  Metropolis - Hastings (d) [  |rest ] ~ [  | ^obs ,  , 

 Inverse Wishart

Results

(based on a regular grid of m=25 sites) Histogram of the parameters

_ _

_ _

_ _

_

_i = _i^-2

Spatial Interpolation

Interpolation grid: s₁ⁿ , ... , s_gⁿ

_jⁿ = (_j(s₁ⁿ) , ... , _j(s_gⁿ) ), j=0, 1, ... , p

ⁿ = (₀ⁿ , ... , _pⁿ )

We need to obtain the interpolation of _j´s to interpolate Yⁿ

Interpolation of Y´s

(Yⁿ,ⁿ,| Y^obs) = (Yⁿ|ⁿ, , Y^obs) (ⁿ,| Y^obs) = (Yⁿ| ⁿ ,) (ⁿ,| Y^obs)

Simulation of [Yⁿ |Y^obs] also in 2 steps:

(a) [ ⁿ, | Y^obs ]  MCMC and Spatial Interpolation (b) [ Yⁿ| ⁿ ,]  using Multivariate Normal

Spatial Interpolation of ´s

(ⁿ,^obs,| Y^obs) = ( ⁿ | ^obs, , Y^obs) ( ^obs, | Y^obs) = ( ⁿ | ^obs ,) ( ^obs, | Y^obs)

Simulation of [ ⁿ | Y^obs ] in 2 steps:

(a)[ ^obs, | Y^obs ]  using MCMC

(b)[ ⁿ | ^obs ,]  using Multivariate Normal

Simulated data: Interpolation of ₁ Simulated values

Interpolated values

0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

0.1 0.3 0.5 0.7 0.9 0.1

0.3 0.5 0.7 0.9

Coordenada 2

Simulated data: Interpolation of Y₃₀(.) Simulated values

Interpolated values

0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

Interpolation of X´s

These interpolations assume that the interpolated covariates X_j are available for j=1, ... , p

Otherwise, we must interpolate them

Simulation of [Xⁿ|Y^obs,X^obs] in 2 steps:

(a) [_x | X^obs ]  MCMC

(b) [Xⁿ| _x, X^obs ]  using Multivariate Normal Model may be completed with

X(.) | _x , _x , _x ~ GP(_x, _x ,_x(.))

(Xⁿ, _x | Y^obs , X^obs) = (Xⁿ , _x| X^obs )

= (Xⁿ| _x, X^obs) (_x | X^obs )

Results obtained by interpolating X Histogram of the parameters

_ _

_ _

_

_

_

_ _

Precisions less sparse then when X is known

Interpolation of X₃₀(.)

0.1 0.3 0.5 0.7 0.9

lat 0.1

0.3 0.5 0.7 0.9

long

0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

Simulated values

Interpolated values

Interpolation of Y₃₀(.)

Known X

Unknown X

0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

0.1 0.3 0.5 0.7 0.9

Coordenada 1 0.1

0.3 0.5 0.7 0.9

Coordenada 2

113.2

Application to the pollution data

_t(s) independents N(0,_²)

₀~ N(_, _,_(.

₁~ N(_, _,_(.

_i(., i=0,1 are exponential correlation functions Y_t (s) = square root of PM10 at site s and time t X_t = (MON, TUE, WED, THU, FRI, SAT)

Y_t(s) = ₀ (s)+ ₁(s)TEMP_t + ´ X_t _t(s)

• Now, the temperature coefficient varies in space

Results for the pollution data in Rio

_

_

_

_

_

_

Histogram of the hiperparameters sample

where _i = _i ^-2

Interpolation of the _coefficient Prior for 

G(10,10c)

SSDE = 0.0637

G(10^-3,10^-3)

SSDE = 0.1444







m



i

ols i obs

i

Y

s m

E

}

] ˆ

| ) ( [ 1 {

where SSDE =

Same idea can be used for (explanatory geostatistical analysis)

c  obtained by exploratory analysis site by site (OLS)

Y_t(s)= _t(s) + e_t(s) where

_t(s)=_t0(s)+_t1(s)X_t1(s)+...+_tp(s)X_tp(s) e_t(s) ~ N(0, _e²) independent

Extension of the previous model Y_t(s)= _t(s) + e_t(s) where

_t(s)= ₀(s)+ ₁(s)X_t1(s)+...+ _p(s)X_tp(s) e_t(s) ~ N(0, _e²) independent

previous model

Natural specification

_t(.) | _t ~ GP(_t , ,_), independent in time The model must be completed with:

(a) prior for (_e ,  ,  as before

(b) specification of the temporal evolution of the _t´s

We can also accommodate temporal variation of the coefficients

_j, j=0,...,p.

Suggestion - use dynamic models (SVP/TVM) (Landim & Gamerman, 2000)

_t | _t-1 ~ N( G_t _t-1 , W_t )

  unknown parameters of the  evolution Model parameters: ^obs , , 

 = ( ₀ , _e² ,  ,  , W ) where

 = ( ₁, ... , _T) and

_t = ( _t0 , _t1, ... , _tp ), t=1, ... , T Simulation cycle has 2 changes:

I) additional step to  II) modified step to 

Application to simulated data

Y_t (s) = _t0 (s) + _t1(s)X_t1(s) + _t(s)

_t(.) ~GP (_t,  ,_)

_t = _t-1 + _t

same spatial correlation to ₀ and ₁

(. exponential correlation function with = 1.

Histogram of the posterior of 

Multivariate observations: Y_t (s) = (Y_t1 (s), Y_t2 (s))

Trajectory of (t) - mean and credibility limits

Interpolation

Samples from y_tⁿ|y^obs are obtained through the algorithm below:

1. Sample from _t^obs, y^obs - through MCMC

2. Sample from _tⁿ_t^obs- through Gaussian process 3. Sample from y_tⁿ_tⁿ - Independent Normal draws

Once again, X_tⁿ must be known, otherwise, they will have to be interpolated.

Spatially- and time-varying parameters (STVP)

Not separable at the latent level, unlike SVP/TVM

Another possibility: temporal evolution applied directly to the _t processes rather than to their means

Y_t (s) = _t0 (s) + _t1(s)X_t1(s) + _t(s)

_t(.) = _t-1(.) + w_t(.)

w_t(.) ~ GP (_t, ,_) independent in time

SVP/TVM:

|) (|

) (

] } ,

min{

[ )]

( ),

(

cov[

2

1

s

s R t t I t t s s

t

       



Completed with: ₀(.) = ₀ ~ N(g₀,R)

STVP:

|)]

(|

}[

, min{

)]

( ),

(

cov[ 

s



s

 R  t

t

W   

s

 s

Marginal Prior:_t(.)| _t ~ GP (_t, t ,_)

Computations

MCMC algorithm must explore the correlation structures

 parameters are visited in blocks

(Landim and Gamerman, 2000; Fruhwirth-Schnatter, 1994)

Based on the forecast distribution of Y_T+h|Y^obs,

for Y_T+h = (Y_T+h(s₁^f ),..., Y_T+h(s_F^f )), and any collection (s₁^f,..., s_F^f)

1. Sample from _T^obs, Y^obs - through MCMC

2. Sample from _T+h^f_T^obs- obtained by introduction of _T+h^obs

3. Sample from Y_tⁿ_Tⁿ - Independent normal draws

_T^obs _T+h^obs by successive evolution of the process

_T+h^obs  _T+h^fby interpolation with gaussian process

Prediction

Time-varying locations

Assume locations s_t = (s^t₁,..., s^t_nt) at time t

_t^obs is a n_t-dimensional vector, t = 1,...,T

1 1

1 1

) ~ ,

~ | , (

) ,

|

( 



   ^p 





 ^d 

p

1 1

1 1

) ~ ,

~ | ( )

~ ,

|

(  

 



 

  ^p

^p

^d

Both densities in the integrand are multivariate normal

The convolution of these two densities can be shown to be normal and required evolution equation for can be obtained SVP/TVM: Easily adapted

STVP: requires introduction of for updated locations

~



Non-Gaussian Observations

Two distinct types of non-normality data:

• Count data:

• Continuous:

Can be normalized after suitable transformation g_(y) Example: Rio pollution data

( g ( Y )  Y )

estimated jointly with other model parameters (de Oliveira, Kedem and Short, 1997)

For example, in the bernoulli or poisson form standard approach: y_t(s) ~ EF(_t(s))

spatio-temporal modeling issues: similar computations: harder

Non-Gaussian Evolution

Abrupt changes in the process  normality is not suitable Robust alternative:

w_t(.) ~ GP(_t,,_) is replaced by

w_t(.)| _t ~ GP(_t,_t^-1, _) and _t ~ G(_t, _t), independent for t=1,...,T

Therefore, w_t(.) ~ tP(_t,,_)

_t’s control the magnitude of the evolution

Final Comments

• More flexibility to accommodate variations in time and space.

• Static coefficient models: samples from the posterior were generated in the software BUGS, with interpolation made in FORTRAN

• Extension to accommodate anisotropic processes to some components of the model.

• Extensions to observations in the exponential family and estimation of the normalizing transformation.

A Latent Approach to the Statistical Analysis of Space-time Data

Dani Gamerman

Instituto de Matemática

Universidade Federal do Rio de Janeiro Brasil

http://acd.ufrj.br/~dani

17th International Workshop on Statistical Modelling Chania, Crete, Greece, 8-12 July 2002