TITLE - Subtitle - MAI - Linköpings universitet

(1)

Explicit Estimators for a Banded Covariance Matrix in a Multivariate Normal Distribution

Presentation

Authors: Emil Karlsson, Martin Singull

Matematiska institutionen Link ¨opings universitet

August 25, 2014

(2)

Background

Historical

History

Patterned covariance matrices Banded covariance matrices

Methods: explicit, maximum likelihood and back again

Authors: Emil Karlsson, Martin Singull Explicit Estimators for a Banded Covariance Matrix

(3)

Background

Notation

List of symbols

A_m,n- matrix of sizem×n

Mm,n- the set of all matrices of sizem×n

a_ij - matrix element of thei-th row andj-th column a_n- vector of sizen

c- scalar

X - random matrix x - random vector X - random variable

(4)

Explicit Estimator

Previous results

Proposition 1

LetX ∼N_p,n(µ1⁰_n,Σ^(m)_(p),I_n). Explicit estimators are given by

ˆ µ_i = 1

nx⁰_i1n,

ˆ σ_i,i = 1

nx⁰_iCx_i fori =1, . . . ,p, ˆ

σ_i,i+1= 1

nˆr⁰_iCx_i+1fori=1, . . . ,p−1,

whereˆr₁=y₁andˆr_i =x_i−sˆ_iˆr_i−1fori =2, . . . ,p−1, ˆs_i = ˆr⁰_i−1Cx_i

ˆr⁰_i−1Cx_i−1, whereC =I_n− 1

n1_n1⁰_n.

(5)

Explicit Estimator

Previous results

Theorem 1

The estimatorµˆ= (ˆµ₁, . . . ,µˆ_p)⁰ given in Proposition 1 is unbiased and consistent, and the estimatorΣˆ^(m)_(p) = (ˆσ_ij)is consistent.[?]

(6)

Explicit Estimator

Purpose of this work

Goals:

Find an unbiased estimator for the covariance matrix.

Generalize results into a general linear model.

Limitations:

Study the case whereΣ^(p)₍₁₎instead ofΣ^(p)_(m).

(7)

Find an unbiased estimator

Rewriting of estimator

Proposition 2

LetX ∼Np,n(µ1⁰_n,Σ⁽¹⁾_(p),In). Explicit estimators are given by

. . . ˆ

σi,i+1= 1

nx⁰_iA_i−1x_i+1fori=1, . . . ,p−1, whereA_i =C−Cˆr_i(ˆr⁰_iCˆr_i)⁻¹ˆr⁰_iC,

withA₀=C,

whereˆr₁=y₁andˆr_i =x_i− ˆr⁰_i−1Cx_i

ˆr⁰_i−1Cx_i−1ˆri−1fori =2, . . . ,p−1, withC=In−1

n1n1⁰_n.

(8)

Find an unbiased estimator

Bilinear form

Definition 1

Letx ∼N_n(µ_x,I_n),y ∼N_n(µ_y,I_n)andA∈M_n,n. Thenx⁰Ay is called a bilinear form.

Theorem 2

The bilinear formx⁰Ay has the following properties.

(i) E[x⁰Ay] =tr(Acov(x,y))

(ii) var[x⁰Ay] =tr(Acov(x,y))²+tr(Avar(x)Avar(y)) = tr(A)cov(x,y)²+tr(A²)var(x)var(y).

(9)

Find an unbiased estimator

Properties of the central matrix

The central matrix forσˆi,i+1= ¹_nx⁰_iA_i−1x_i+1 A_i =C−Cˆr_i(ˆr⁰_iCˆr_i)⁻¹ˆr⁰_iC Properties:

Idempotent,A²_i =A_i Symmetric,A⁰_i =A_i

(10)

Find an unbiased estimator

Unbiased estimator

Proposition 3

LetX ∼N_p,n(µ1⁰_n,Σ⁽¹⁾_(p),I_n). Explicit estimators are given by

ˆ

σ_ii = 1

n−1x⁰_iCx_i fori =1, . . . ,p, ˆ

σ₁₂ = 1

n−1x⁰₁Cx₂, ˆ

σ_i,i+1= 1

n−2x⁰_iA_i−1x_i+1fori =2, . . . ,p−1, whereA_i =C−Cˆr_i(ˆr⁰_iCˆr_i)⁻¹ˆr⁰_iC, withA₀=C,

whereˆr₁=y₁andˆr_i =x_i− ˆr⁰_i−1Cx_i

ˆr⁰_i−1Cx_i−1ˆr_i−1fori =2, . . . ,p−1, withC=I_n−1

n1_n1⁰_n.

(11)

Find an unbiased estimator

Results

Theorem 3

The estimators from Proposition 3 are unbiased and consistent.

Variance is known:

var(ˆσ_i,i+1) = ^σ

2

i,i+1+σiiσi+1,i+1

n−2

(12)

Generalization to a general linear model

General linear model

Assumptions:

General linear model

Y =X B+E ∼N_n,p(X B,I_n,Σ^(p)₍₁₎)

Y andE aren×mrandom matrices

X is a knownn×p-design matrix with full rank

Bis an unknownp×m-matrix of regression coefficients.

n≥m+p, and the rows of the error matrixE are independentN_m(0,Σ)random vectors.

(13)

Generalization to a general linear model

Problems

Problems when generalizing

The expected valueX Bdiffers fromµ1⁰.

The design matrix affects the degrees of freedom.

(14)

Generalization to a general linear model

Expected value

Transformation:

(Y −X B)∼N(0,I_n,Σ)can be treated as

(y_i−X b_i)∼N(0,Σ)were each part is handled separately

(15)

Generalization to a general linear model

Unbiased version

Proposition 4

LetY =X B∼N_p,n(X B,Σ⁽¹⁾_(p),I_n), where rank(X) =k . Explicit estimators are given by

Bˆ = (X⁰X)⁻¹X⁰Y, ˆ

σ_ii = 1

n−ky⁰_iDy_ifori=1, . . . ,p, ˆ

σ_i,i+1= 1

n−k−1y⁰_iA_i−1x_i+1fori =2, . . . ,p−1, whereA_i =D−Dˆr_i(ˆr⁰_iDˆr_i)⁻¹ˆr⁰_iDwithA₀=D,

whereˆr₁=y₁andˆr_i =y_i− ˆr⁰_i−1Dy_i

ˆr⁰_i−1Dy_i−1ˆr_i−1fori =2, . . . ,p−1, withD=I_n−X(X⁰X)⁻¹X⁰.

(16)

Generalization to a general linear model

Variance

Theorem 4

The estimators from Proposition 4 are unbiased and consistent.

Known variance:

var(ˆσ_i,i+1) = 1

n−2(A)(σ²_i,i+1+σ_i,iσ_i+1,i+1)

(17)

Simulations

Normal case

Based on the 100000 averages of samples with n=20, explicit unbiased average estimators, with true value within

parenthesis, are given by,

Σˆnew =







4.99501(5) 1.99590(2) 0.00000 0.00000 1.99590(2) 4.99238(5) 0.99678(1) 0.00000 0.00000 0.99678(1) 5.00026(5) 3.00265(3) 0.00000 0.00000 3.00265(3) 5.00368(5)





 ,

and the previous estimators are given by,

Σˆprev =







4.74526(5) 1.89611(2) 0.00000 0.00000 1.89611(2) 4.74276(5) 0.89710(1) 0.00000 0.00000 0.89710(1) 4.75025(5) 2.70239(3) 0.00000 0.00000 2.70239(3) 4.75350(5)





 .

(18)

Simulations

General linear model-case

Based on the 100000 averages of samples with n=80 and 20 regression parameters, whereXandBwere randomly

generated, unbiased explicit average estimators, with true value within parenthesis, are given by,

Σˆnew =







3.9986(4) 0.9997(1) 0 0 0

0.9997(1) 3.0051(3) 2.0024(2) 0 0 0 2.0024(2) 4.9989(5) 2.9976(3) 0 0 0 2.9976(3) 4.9941(5) 2.9950(3)

0 0 0 2.9950(3) 4.9935(5)





 ,

and the previous estimators are given by,

Σˆ_prev =







2.9989(4) 0.7497(2) 0 0 0

0.7497(2) 2.2538(3) 1.4768(2) 0 0 0 1.4768(2) 3.7492(5) 2.2107(3) 0 0 0 2.2107(3) 3.7455(5) 2.2088(3)

0 0 0 2.2088(3) 3.7451(5)





 .

(19)

Simulations

Results

Conclusion:

The unbiased version makes an considerable improvement

(20)

Discussion

Further reasearch

Topics:

Find unbiased estimator forΣ^(p)_(m)

Compare it to other estimators(for example MLE) for banded matrices.

Study the variance to determine efficiency

(21)