Diagnostic and treatment for linear mixed models Julio M Singer

(1)

Diagnostic and treatment for linear mixed models

Julio M Singer

in collaboration with

Francisco M.M. RochaandJuvˆencio S Nobre Departamento de Estat´ıstica

Universidade de S˜ao Paulo, Brazil www.ime.usp.br/∼jmsinger

(2)

Outline

Two examples

Gaussian linear mixed models (LMM) Diagnostic tools

Residual analysis Global influence analysis Local influence analysis Treatment

Fine tuning of the model

LMM with elliptically-symmetric random effects LMM with skew-symmetric random effects Generalized linear mixed models (GLMM) Generalized estimating equations based models

Practical issues

Where do we go from here

(3)

Ozone example

Ozone concentration: measured with expensive instruments Alternative: reflectance in passive filters / calibration curve

(4)

Ozone example

Experiment LPAE/FMUSP: predict period expected reflectance (latent value) accounting for possible outliers

Period Reflectance Period Reflectance

1 27.0 6 47.9

1 34.0 6 60.4

1 17.4 6 47.3

2 24.8 7 50.4

2 29.9 7 50.7

2 32.1 7 55.9

3 35.4 8 54.9

3 63.2 8 43.2

3 27.4 8 52.1

4 51.2 9 38.8

4 54.5 9 59.9

4 52.2 9 61.1

5 77.7

5 53.9

5 48.2

(5)

Ozone example

Linear mixed model:

yij =µ+ai+eij, i= 1, . . . ,9, j= 1,2,3

a_i∼N(0, σ²_a)independent eij∼N(0, σ²)independent a_i ande_ij independent

Consequently

V(yij) =σ²_a+σ² Cov(yij, yik) =σ_a² Cov(y_lj, y_ik) = 0

Reliability of the mean: ρm=σ_a²/(σ_a²+σ²/3)

(6)

Kcal intake example

Study conducted at FMUSP

Compare average daily kcal intake during pregnancy

Após a exclusão das gestantes com consumo maior que 4000 Kcal:

4000 3000 2000 1000 0

3 2

1 4000 3000 2000 1000 0

3 2

1 1

Trimestre

Energia

2

3 4

Panel variable: IMC_C

Valores de estatísticas descritivas para Energia

Trimestre N Média Desvio padrão Mínimo Mediana Máximo 1 160 1973,2 689,5 584,4 1903,7 3905,0 2 146 2148,8 598,1 699,4 2107,0 3754,1 3 135 2124,8 649,4 509,2 2038,5 3820,2

(7)

Kcal intake example

Linear mixed model

y_ijk=µj+αi+a_ijk+e_ijk, α1 = 0

iindexesBM I (1 =BM I <24.9kg/m²,2 =BM I≥24.9kg/m²) j indexes period (1 = 1st trimester and2 = 2ndor 3rdtrimesters) kindexes women (k= 1, . . . , ni)

bik= (ai1k, ai2k, ai3k)^>∼N3(0,G) independent

G=





σ²₁ σ12 σ13

σ₁₂ σ²₂ σ₂₃ σ₁₃ σ₂₃ σ₃²





eij = (ei1k, ei2k, ei3k)^>∼N3[0, σ²I3] independent bik and eik independent

(8)

Preterm neonates example

Aorta diameter per unit weight (mm/kg)

Weeks post conception

Weight 27 28 29 30 31 32 33 34 35 36 37 38 39 40

AGA 9.4 10.3 8.7 8.2 6.7 6.1 5.6 5.1 4.9

AGA 6.1 6.1 6.2 5.4 5.2 4.9

AGA 5.8 6.3 5.8 5.1 4.9 4.6

AGA 9.7 9.2 9.5 7.3 6.1 5.4 4.8 4.5

AGA 6.4 5.8 5.2 4.7

AGA 5.4 4.9 4.6 4.3

AGA 8.3 8.5 8.6 7.9 6.2 5.5 4.2

AGA 7.7 8.6 7.9 6.6 5.7

AGA 5.9 6.1 5.4 4.1

AGA 7.0 6.5

AGA 5.2 4.8 4.2 4.1 3.7

. . . . . . . . . . . . . . .

AGA 6.2 6.1 6.2 6.0 5.3

SGA 7.2 6.8 5.5 4.7

SGA 7.1 8.0 7.7 6.5 5.6

SGA 7.4 8.3 9.4 10.0 9.2 8.0

SGA 7.7 6.6 5.5 4.6

SGA 6.5 4.4

SGA 7.6 8.6 9.3 8.0 6.6 5.0 4.7

SGA 6.6 8.4 8.2 7.6 6.6

SGA 7.1 6.3 6.1 5.9 5.7 4.8

SGA 8.5 8.4 4.9

SGA 8.3 7.4 6.2 4.6 3.8

SGA 9.8 9.1 7.3 5.3

. . . . . . . . . . . . . . .

SGA 8.5

SGA 10.9 10.7 9.4 8.0 5.8 4.9

(9)

Preterm neonates example

Profile plots for the Preterm neonates example

Mean Profile in bold

Aorta diameter (mm/kg)

4 6 8 10

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 AGA

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 SGA

Loess in bold Weeks

Aorta diameter (mm/kg)

4 6 8 10

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 AGA

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 SGA

JM Singer (USP) EMR 2013, Maresias, SP 9 / 64

(10)

Preterm neonates example

Objective: evaluate evolution of aorta diameter of preterm neonates from birth to 40-th week post conception

Linear mixed model suggested via exploratory analysis (Rocha &

Singer, 2013, under revision)

y_ijk=α_i+β₁(t_1jk−26) +γ₂(t_2jk−26)²+a_ij+b_ij(t_ijk−26) +e_ijk iindexes group (1=AGA and 2=SGA)

j indexes neonates (j= 1, . . . , ni) kindexes week (k= 1, . . . , mij)

bij = (aij, bij)^>∼N2(0,Gi) independent G_i=

σ²_a_i σabi

σ_ab_i σ²_b

i

eij = (eij1, . . . , eijmij)^>∼Nmij[0, σ²Imij]independent bij andeij independent

(11)

Gaussian LMM

y_i =X_iβ+Z_ib_i+e_i, i= 1, . . . , n yi: (mi×1) response profile fori-th unit

β: (p×1) (fixed effects)

Xi: (mi×p) fixed effects specification matrix Z_i: (m_i×q) random effects specification matrix b_i: (q×1) random effects,b_i∼N_q(0,G) independent ei: (mi×1) random errors,ei ∼Nmi(0,Ri) independent bi andei independent

G=G(θ)and Ri=Ri(θ),θ: covariance parameters Marginal variance: V(y_i) =V_i =Z_iGZ^>_i +R_i

Usually R_i=σ²I_m_i: homoskedastic conditional independence model

(12)

Gaussian LMM

Compactly

y=Xβ+Zb+e with

y= (y₁^>,· · ·,y_n^>)^> (N ×1, N =Pn i=1m_i) X= (X^>₁,· · ·,X^>_n)^> (N ×p)

Z=⊕ⁿ_i=1Zi (N ×nq) b= (b^>₁,· · · ,b^>_n)^> (nq×1) e= (e^>₁,· · · ,e^>_n)^> (N×1) Γ=I_n⊗G(θ) (nq×nq) Σ=⊕ⁿ_i=1Ri(θ) (N ×N) Consequently

V(y) =V=ZΓZ^>+Σ

(13)

Covariance matrix structure

1 Uniform

R_i(θ),G(θ) =





σ² τ τ τ σ² τ τ τ σ²





2 Unstructured

R_i(θ),G(θ) =





σ²₁ σ12 σ13

σ₁₂ σ₂² σ₂₃ σ13 σ23 σ₃²





3 AR(1)

R_i(θ),G_i(θ) =σ²





1 φ φ²

φ 1 φ

φ² φ 1





(14)

Inference for Gaussian LMM

Given θ [Γ(θ),Σ(θ) andV(θ)]

BLUEof β: βb = X^>V⁻¹X−1

X^>V⁻¹y BLUP of b: bb=ΓZ^>V⁻¹[I−X X^>V⁻¹X−1

X^>V⁻¹]y Ozone example BLUP

ybij=y+k(y_i−y) Shrinkage constant

k= σ²_a σ_a²+σ²/3

SubstitutingΓb andΣb in the expressions forβb and bb we obtain empirical BLUE and BLUP

(15)

Inference for Gaussian LMM

Restricted maximum likelihood (REML) for θ (Γ, Σ, V):

−1 2

N

X

i=1

tr{V⁻¹_i (bθ) ˙Vi(bθ)} − 1 2

N

X

i=1

[∂Qi(θ)/∂θj|_θ=b_θ]

−1 2

N

X

i=1

tr{V⁻¹_i (bθ)X^>_i V⁻¹_i (bθ) ˙V_i(bθ)V⁻¹_i (bθ)X_i}= 0,

V˙i(bθ) = [∂Vi(θ)/∂θj]^>

θ=bθ

Qi(θ) = [yi−Xiβ(θ)]b ^>V⁻¹_i (θ)[yi−Xiβ(θ)]b Newton-Raphson algorithm:

θ^(l)=θ^(l−1)−H⁻¹[θ^(l−1)]u[θ^(l−1)], l= 1,2, . . . Score function: u(θ) =∂l[bβ(θ),θ]/∂θ

Hessian matrix: H(θ) =∂²l[bβ(θ),θ]/∂θ∂θ^>

Stopping rule: kθ^(l)−θ^(l−1)k< ε,ε >0

Fisher Scoringalgorithm: H(θ) replaced by its expectation EM algorithm

(16)

Statistical properties of estimators

β(θ)b ∼N{β,V_β_b(θ)}with V_β_b(θ) = [PN

i=1X^>_i V⁻¹_i (θ)Xi]⁻¹ bθ≈N{θ,V

bθ(θ)}with V

bθ(θ) denoting an (m×m) matrix for which the element (r, s), r, s= 1, . . . , m, is

[Vbθ(θ)]_rs= ¹₂PN

i=1tr{V⁻¹_i (θ) ˙V_ir(θ)V_i⁻¹(θ) ˙V_is(θ)}

β(b bθ)≈N{β,V

βb(θ)}

Vβb(θ) andV

bθ(θ) may be estimated by the inverse of Fisher’s information matrix

Asymptotic results hold even without normality provided nis sufficiently large

(17)

Types of residuals in Gaussian LMM

Three types of residuals that accommodate theextra sourceof variability present in linear mixed models, namely:

i) Marginal residuals,bξ=y−Xbβpredictors of marginal errors, ξ=y−E[y] =y−Xβ=Zb+e

ii) Conditional residuals,be=y−Xbβ−Zbbpredictors ofconditional errorse=y−E[y|b] =y−Xβ−Zb

iii) BLUP,Zbb, predictors ofrandom effects,

Zb=E[y|b]−E[y] = (y−Xβ−Zb)−(y−Xβ)

(18)

Confounded Residuals

Hilden-Minton (1995, PhD thesis, UCLA): residual is purefor an error if it depends only on fixed components and on error it is supposed to predict

Otherwise: confoundedresiduals Given that

bξ = [I−X(X^>Vb⁻¹X)⁻¹X^>Vb⁻¹]ξ, be = ΣbQeb +ΣbQZb,b

Zbb = ZbΓZ^>QZbb +ZbΓZ^>Qe,b whereQ=V⁻¹−V⁻¹X X^>V⁻¹X−1

X^>V⁻¹, we have beis confoundedwith bb

Zbb isconfounded withbe

Exception: columns ofZbelong to the space generated by the columns of X

(19)

Marginal Residuals

Sincey=Xβ+ξ, plots of marginal residuals (bξ_ij) versus explanatory variables or fitted values (ybij =x^>_ijβ) tob check for linearity

Index plots ofbξ_ij todetect outlying observations

Lesaffre and Verbeke (1998, Biometrics): (unit) index plots of Ri =Vb^−1/2_i bξ_i useful tocheck appropriateness of the within-unit covariance matrix

WhenVi=||Im_i−RiR^>i ||²is small, within-units covariance matrix is acceptable for uniti [E(bξ_ibξ^>_i ) =Vb_i ??]

In lieu ofVb_i, we suggest usingVbi(ξ), thei-th diagonal block of

Vb(bξ) = [Vb −X(XVb⁻¹X)⁻¹X^>] Also, we suggest using (unit) index plots ofV^∗i =Vi/Pn

i=1Vi to allow comparison among different models

(20)

Conditional Residuals

Conditional studentized residuals (Nobre and Singer, 2007, Biometrical Journal): be^∗_ij =be_ij/p

pb_ij

p_ij: ij-th element of the main diagonal ofVb(be) =ΣbQbΣb

We suggestbe^∗_i = [Vbi(be)]^−1/2ebi where Vbi(be)is thei-th block ofVb(be) Index plot ofeb^∗_ij to detect outlying observations

Plot of be^∗_ij versus predicted values (yb^∗_ij =x^>_ijβb+z^>_ijbb_i) tocheck for homoskedasticity of conditional errors: (Ri =σ²Imi)

Check for normality of conditional errors Must take confounding into consideration

bemay not be adequate to check for normality ofe

Whenbis non-normal,bemay not be normal even wheneis

(21)

Conditional Residuals

Hilden-Minton (1995, PhD thesis, UCLA): ability to check for normality of e, using be,decreasesasV[ΣQZ^>b] =ΣQZΓZ^>QΣ increases in relation to V[ΣQe] =ΣQΣQΣ

Fraction of confounding for the ij-th conditional residualbe_ij

0≤F_ij = u^>_ijΣQZΓZ^>QΣuij

u^>_ijΣQΣu_ij = 1− u^>_ijΣQΣQΣuij

u^>_ijΣQΣu_ij ≤1 whereuij is ij-th column of IN

Least confounded residuals: linear transformationCbethat maximizes λ_ij = c^>_ijΣQΣQΣcij

c^>_ijΣQΣc_ij , i= 1, . . . , n, j= 1, . . . , m_i

Least confounded residuals: homoskedastic, uncorrelated, varianceσ² QQ plots and histograms to check for normality of conditional residuals

(22)

EBLUP

EBLUP: reflects the difference between the predicted expected response (latent value) for i-th unit and population average Unit index plots of Mi =bb^>_i {Vb[bb_i]}⁻¹bb_i to detect outlying units We suggest (unit) index plots of M^∗_i =Mi/Pn

i=1Mi to allow comparison among different models

To assess normality of random effects:

No confounding: χ²_q QQ plot of Mi

With confounding: ?

(23)

Residual diagnostics for Gaussian LMM

Diagnostic for Residual Plot

Linearity of effects fixed (E[y] =Xβ) Marginal bξ^∗_ij vsfitted values or explanatory variables Presence of outlying observations Marginal bξ^∗_ij vsobservation indices Within-subjects covariance matrix (Vi) Marginal V^∗i vsunit indices Presence of outlying observations Conditional be^∗_ij vsobservation indices Homoskedasticity of conditional errors (ei) Conditional be^∗ij vspredicted values Normality of conditional errors (ei) Conditional Gaussian QQ plot forbe^∗_ij

orc^>_ijbe^∗

Presence of outlying units EBLUP M^∗i vsunit indices Normality of the random effects (bi) EBLUP χ²q QQ plot forMi

(24)

Global influence analysis (leverage)

Generalized leverage matrix (forfixed effects), by=Xbβ L1 = ∂by

∂y^> =X

X^>V⁻¹X−1

X^>V⁻¹, tr(L1) =p High leverage unit: tr(L_1i)/m_i >2p/n whereL_1i isi-th block ofL₁ High leverage observation: L_1ijj >2p/N whereL_1ijj isj-th diagonal element ofL1i

Since by^∗=Xβb+Zbb: Generalizedjoint leverage matrix L= ∂yb^∗

∂y^> = ∂yb

∂y^> + ∂Zbb

∂y^> =L1+ZΓZ^>Q L₂ =ZΓZ^>: variability explained by random effects

Demidenko and Stukel (2005, SIM) suggest using H₂=ZΓZ^>Q as generalized leverage matrix for random effects

SinceH2 =L2V⁻¹[In−L1], Nobre and Singer (2011, JAS) argue for L₂ =ZΓZ^>

(25)

Global influence analysis (case deletion)

Impact of a unit on some characteristic (e.g., parameter estimate) βb−βb_(I)= (X^>V⁻¹X)⁻¹X^>V⁻¹UI(U^>_IQUI)⁻¹U^>_IQy

Cook´s distance

D_I= (βb−βb_(I))^>(X^>V⁻¹X)(bβ−βb_(I))

p = (yb−yb_(I))^>V⁻¹(yb−yb_(I)) p

Conditional Cook distance (forR_i =σ²I_m_i) D^cond_i(j) =

n

X

i=1

(yb^∗_i −by_i(j)^∗ )^>(by^∗_i −by^∗_i(j)) σ²(n+p)

whereyb_i^∗=Xiβb+Zibb,yb^∗_i(j)=Xiβb_(i(j))+Zibb_(i(j)) Ratio of variance ellipsoids

ρ(I)= V(bb β_(I))

|V(bb β)| =

IN+X^>V⁻¹UI

U^>IQUI

−1

U^>IV⁻¹X

X^>V⁻¹X−1

(26)

Global influence analysis

Diagnostic for effect on Global influence measure Index plot of

Fixed portion of Generalized marginal L1i(jj) [tr(L1i)/mi]vs fitted value (Xbβ) leverage matrixL1 observations (units) Random portion of Generalized random component L2i(jj) [tr(L2i)/mi]vs fitted value (Zbb) marginal leverage matrixL2 observations (units) Regression coefficients (β)b Cook’s distanceDI Di(j) [Di]vs

observations (units) Covariance matrix of Ratio of variance ρ_i(j) [ρi]vs regression coefficients [V(bb β)] ellipsoidsρ(I) observations (units)

Diagnostic for effect on Index plot of

Regression coefficients (bβ) D_1i(j)^condvs observations Random effects (bb) D_2i(j)^condvs observations Changes in covariance

betweenβbandbb D_3i(j)^condvs observations

(27)

Local influence analysis

Changes in the analysis resulting fromsmall perturbationson the data or on some element of the model

Behaviour oflikelihood displacement: LD(ω) = 2n

L(bψ)−L(bψ_ω|ω)o L: likelihood for proposed model

ψ: parameter vector ω: vector of perturbations

ψb andψb_ω: MLE ofψ based onL(ψ)andL(ψ|ω) Usual perturbation schemes

Response variables: yi(ωi) =yi+ωi

Explanatory variables: Xi(Wi) =Xi+Wi

Random effects covariance matrix: G(ωi) =ωiG Error covariance matrix: Ri(ωi) =ωiRi

Index plots of normalized eigenvectors corresponding to the largest eigenvalue of−H^>F¨⁻¹Hwhere F¨ = [∂²L(ψ)/∂ψ^>∂ψ]_ψ=

ψb and H= [∂²L(ψ|ω)/∂ψ^>∂ω]_ω=ω

0;ψ=ψb suggest units or observations with greater impact

(28)

Treatment: fine tuning of model

Diagnostic tools depend oncorrect specification of covariance structure

Examination of linear models fitted to individual profiles (or to rows of within-unit covariance matrix when available) [Rocha and Singer (2012, submitted)]

Use of simplet-tests

Coefficients significantly different from zero are candidates for random effects

Plots of covariances and correlationsversus lags (Grady and Helms (1995, SIM): useful to identify auto-regressive structures

Modelling covariance structure as function of explanatory variables [Singer and C´uri (2006, Environ Ecol Stat)]

(29)

Treatment: heavy-tailed distributions

Elliptically-symmetric distributions Useful to accommodate outliers

Density function: f(y) =|Σ|^−1/2g[(y−µ)^>Σ⁻¹(y−µ)]

gnon-negative valued function E(y) =µ,V(y) =Ω=αΣ

αconvenient constant (=ν/(ν−2), ν >2for multivariate-twithνdf) Includesmultivariate-t, slash, contaminated normaletc

LMM may be defined hierarchically

yi|bi∼ESm_i[Xiβ+Zibi,Ri(θ), αi, γi] bi∼ESq[0,G(θ), αi]

ei∼ESm_i[0,Ri(θ), γi]

Joint distribution of(y^>_i ,b^>_i )^> may not belong to the same class Maximum likelihood estimation similar to Gaussian case (but more problematic)

Asymptotic properties of estimators still deserves study Local influenceconsidered by Osorio et al. (2007, CSDA) Residual analysis andglobal influence analysis ?

(30)

Treatment: asymmetric distributions

Skew-ellipticaldistributions

Fitting of general case is difficult in practice [Jara et al. (2008, CSDA)]

In general: Bayesian methods

Alternative: skew-normal hierarchical LMM

y_i|[β,b_i,R_i(θ),Λ_e_i]∼SN_m_i[X_iβ+Z_ib_i,R_i(θ),Λ_e_i] b_i|[G(θ),Λ_b]∼SN_q[0,G(θ),Λ_b]

Λ_e_i andΛ_b are asymmetry parameters

Asymptotic properties of estimators still deserves study

Local influencebased on EM algorithm considered by Bolfarine et al.

(2007, Sankhya)

Residual analysis andglobal influence analysis ?

(31)

Treatment: GLMM models

GLMM allow non-normal distributions and non-linear models Models may be defined in a two-stage approach

f(yij|bi) = exp{φ[yijϑij−a(ϑij)] +c(yij, φ)}

aandc: known functions andφ: scale parameter E(yij|bi) =µij=da(ϑij)/dϑij

V(yij|bi) =φ⁻¹d²(ϑij)/dϑ²_ij g(µij) =ηij =x^>_ijβ+z^>_ijbi

Second stage, usually: bi∼N[0,G(θ)]

Fitting is complicated (usually based on EM algorithm) Interpretation of parametersis not straightforward Residual analysis?

Case deletion analysis [Xu et al. (2006, CSDA)]

(32)

Treatment: GEE based models

GEE-based models focus on marginal distributions: not mixed model No need to specifyform of underlying distribution

E(yij) =µij andV(yij) =φ⁻¹ν(µij)

ν(µij): known function of the mean and φ: scale parameter.

Relation between response mean and explanatory variables:

g(µij) =ηij =x^>_ijβ

Working correlation matrix: VW i(θ) =φA^1/2_i RW(θ)A^1/2_i Ai=⊕^m_j=1ⁱ ν(µij)

R_W(θ): known positive-definite matrix Generalized estimating equations

n

X

i=1

X^>_i ∆i[VW i(θ)]⁻¹[yi−µ_i(X^>_i β)] =0

βb asymptotically normal even if working covariance matrix misspecified

Residual analysis: Venezuela et al. (2007, JSCSimulation) Local influence: Venezuela et al. (2011, CSDA)

(33)

Computation

Random effects GorR_W Error R_i

Library Function Fits distribution matrix distribution matrix

lme4 lmer LMM gaussian unstructuredG gaussian σ²I_mi

nlmer NLMM gaussian unstructuredG gaussian structured

glmer GLMM gaussian unstructuredG exponential NA

family

nlme lme LMM gaussian structuredG gaussian structured

nlme NLMM gaussian structuredG gaussian structured

gls LM NA NA gaussian structured

gee gee GEE-based NS structuredR_W exponential NA

model family or NS

geepack geeglm GEE-based NS structuredR_W exponential NA

model family or NS

heavy heavyLme ES-LMM elliptically unstructuredG elliptically NA

symmetric symmetric NA

NA: not applicable NS: not specified

Functions for diagnostic available only from authors Difficult to use in more complicated problems

First version of functions for residual diagnostic based on lme4 and nlme being developed

(34)

Ozone example - standard model (A)

Results (standard model): µb= 46.1,bσ_a² = 100.4,bσ² = 104.8,k= 0.75 Standardized marginal residuals - Ozone standard model

30 40 50 60

−3−2−10123

Marginal fitted values

Standardized marginal residuals

5.1

Density

−2 −1 0 1 2 3

0.000.100.200.30

(35)

Ozone example - standard model (B)

Standardized Mahalanobis distance - Ozone standard model

2 4 6 8

0.00.20.40.6

Units

Standardized Mahalanobis distance

1 2

(36)

Ozone example standard model (C)

QQ plot for Mahalanobis distance - Ozone standard model

0.0 0.5 1.0 1.5 2.0 2.5 3.0

0.00.51.01.52.02.53.03.5

Chi−squared quantiles

Mahalanobis distance

(37)

Ozone example standard model (D)

Standardized LeSaffre-Verbeke measure - Ozone standard model

2 4 6 8

0.00.10.20.30.40.50.60.7

Units

Standardized Lesaffre−Verbeke measure

3

5

(38)

Ozone example standard model (E)

Standardized conditional residuals - Ozone standard model

35 40 45 50 55

−3−2−10123

Predicted values

Standardized conditional residuals

5.1

Density

−2 −1 0 1 2 3

0.000.100.200.30

(39)

Ozone example standard model (F)

Standardized least confounded conditional residuals - Ozone standard model

−2 −1 0 1 2

−1.5−1.0−0.50.00.51.01.5

N(0,1) quantiles

Standardized least confounded residuals

Density

−2 −1 0 1 2

0.00.10.20.30.4

(40)

Ozone example heteroskedastic model (A)

Suggested (heteroskedastic) model

y_ij =µ+a_i+e_ij with e_ij ∼N(0, σ_i²) For parsimony: σ²_i =τ², i= 3,5,σ_i²=σ², otherwise Shrinkage constant: k_i=σ²_a/(σ_a²+σ_i²/3)

Results heteroskedastic model: µb= 46.4,bσ_a² = 114.3,σb² = 49.6, τb² = 274.0,ki6=3,5 = 0.87,ki=3,5 = 0.56

Results homoskedastic model): µb= 46.1,σb²_a= 100.4,bσ²= 104.8, k= 0.75

Predicted latent values

Heteroskedastic model: by_3∗= 43.8, by_5∗= 53.9 Standard model: by_3∗= 42.8, by_5∗= 56.4

(41)

Ozone example heteroskedastic model (B)

Standardized Mahalanobis distance - Ozone heteroskedastic model

2 4 6 8

0.00.20.40.60.8

Units

Standardized Mahalanobis distance

1 2

(42)

Ozone example heteroskedastic model (C)

Standardized Lesaffre-Verbeke measure - Ozone heteroskedastic model

2 4 6 8

0.00.20.40.60.8

Units

Standardized Lesaffre−Verbeke measure

1

9

(43)

Ozone example heteroskedastic model (D)

Standardized conditional residuals - Ozone heteroskedastic model

30 35 40 45 50

−3−2−10123

Predicted values

1.3

Density

−2 −1 0 1 2

0.00.10.20.30.4

(44)

Ozone example heteroskedastic model (E)

Standardized least confounded conditional residuals - Ozone heteroskedastic model

−2 −1 0 1 2

−2−1012

N(0,1) quantiles

Density

−3 −2 −1 0 1 2 3

0.000.100.200.30

(45)

Ozone example heteroskedastic model (F)

Period Reflectance Period Reflectance

1 27.0 6 47.9

1 34.0 6 60.4

1 17.4 6 47.3

2 24.8 7 50.4

2 29.9 7 50.7

2 32.1 7 55.9

3 35.4 8 54.9

3 63.2 8 43.2

3 27.4 8 52.1

4 51.2 9 38.8

4 54.5 9 59.9

4 52.2 9 61.1

5 77.7

5 53.9

5 48.2

(46)

Kcal intake example (A)

Model: yijk=µj+αi+aijk+eijk, α1 = 0

G=





σ₁² σ₁₂ σ₁₃ σ12 σ²₂ σ23

σ13 σ23 σ²₃



, R=σ²I3

Results:

bµ₁ = 2085±62 bµ2 = 2246±52 αb2 =−290±84

(47)

Kcal intake example (B)

1000 2000 3000

−4−2024

Predicted values

1001.2

1003.3 1008.1

1031.1 1041.31051.1 1066.21069.2

1071.1

1079.1

1090.3 1092.2 1099.1

1142.2 1194.1 2033.1 2083.2

2135.2

2145.3 2164.1

2164.2

2173.1 2181.3

Density

−3 −2 −1 0 1 2 3

0.00.10.20.30.4

(48)

Kcal intake example (C)

Standardized least confounded conditional residuals

−3 −2 −1 0 1 2 3

−3−2−10123

N(0,1) quantiles

Density

−3 −2 −1 0 1 2 3

0.00.10.20.30.4

(49)

Kcal intake example (D)

Residual analysis suggests inadequacy of correlation structure for conditional errors

Alternative model:

G=τ²I₃, R=





σ₁² ρσ1σ2 ρσ1σ3

ρσ₁σ₂ σ²₂ ρσ₂σ₃ ρσ1σ3 ρσ2σ3 σ²₃





Results:

bµ1 = 2083±62 bµ2 = 2244±52 αb₂ =−287±84

ρb= 0.36

(50)

Kcal intake example (E)

1800 2000 2200 2400

−4−2024

Predicted values

1001.2

1003.3 1008.1 1031.1

1041.3 1051.1

1066.2 1069.2 1079.1

1090.3 1092.2 1099.1

1142.2 1194.1

2033.1 2083.2

2135.2 2145.32164.1

2164.2 2173.1 2181.3

Density

−3 −2 −1 0 1 2 3

0.00.10.20.30.4

(51)

Kcal intake example (F)

Standardized least confounded conditional residuals

−3 −2 −1 0 1 2 3

−3−2−10123

N(0,1) quantiles

Density

−3 −2 −1 0 1 2 3

0.00.10.20.30.4

(52)

Preterm neonates example

Model:

yijk=αi+β1(t1jk−26) +γ2(t2jk−26)²+aij+bij(tijk−26) +eijk

iindexes group (1=AGA and 2=SGA) j indexes neonates (j= 1, . . . , ni) kindexes week (k= 1, . . . , mij)

b_ij = (a_ij, b_ij)^>∼N₂(0,G_i) independent Gi=

σ²_a_i σ_ab_i σabi σ²_b

i

eij = (eij1, . . . , eijmij)^>∼Nmij[0, σ²Imij]independent bij andeij independent

(53)

Preterm neonates example (A)

4 5 6 7 8

−4−2024

Marginal fitted values

1.1 1.3 4.3

12.2

19.1 28.3

32.4 35.3

53.1 36.1 58.4 58.5

58.7 61.2

Density

−3 −1 0 1 2 3

0.00.10.20.30.4

(54)

Preterm neonates example (B)

Standardized Mahalanobis’s distance

0 10 20 30 40 50 60

0.000.050.100.150.20

Units

Standardized Mahalanobis distance ₄

11 12

28 32

5253 61

(55)

Preterm neonates example (C)

QQ plot for Mahalanobis’s distance

0 2 4 6 8 10 12 14

024681012

Chi−squared quantiles

Mahalanobis distance

(56)

Preterm neonates example (D)

4 6 8 10

−4−2024

Predicted values

1.1 1.3 4.3 12.2

19.1 28.3

32.4 35.3

53.1 36.1

58.4 58.5

58.7 61.2

Density

−3 −1 0 1 2 3

0.00.10.20.30.4