164 · Chapter 2: Regression IMSL STAT/LIBRARY * * * Variance-Covariance Matrix for the Coefficient Estimates * * * 1 2 3 4 5 1 3.57593 0.00000 0.00000 0.00000 0.00000 2 3.57593 -3.57593 0.00000 0.00000 3 3.57593 0.00000 0.00000 4 7.15185 -3.57592 5 7.15185 6 7 8 9 10 1 0.00000 0.00000 0.00000 0.00000 0.00000 2 0.00000 0.00000 0.00000 0.00000 0.00000 3 0.00000 0.00000 0.00000 0.00000 0.00000 4 -3.57593 0.00000 0.00000 0.00000 0.00000 5 -3.57593 0.00000 0.00000 0.00000 0.00000 6 7.15185 0.00000 0.00000 0.00000 0.00000 7 7.15185 -3.57592 -3.57593 -7.15185 8 7.15185 -3.57593 3.57592 9 7.15185 3.57593 10 7.15185 11 12
1 0.00000 0.00000 2 0.00000 0.00000 3 0.00000 0.00000 4 0.00000 0.00000 5 0.00000 0.00000 6 0.00000 0.00000 7 3.57592 3.57593 8 -7.15185 3.57593 9 3.57593 -7.15185 10 -3.57592 -3.57593 11 7.15185 -3.57593 12 7.15185
IMSL STAT/LIBRARY Chapter 2: Regression · 165 S2 — s2, the estimated variance of the error in the regression model. (Input)
s2 is the error mean square from the regression fit.
COVB — NCOEF by NCOEF matrix that is the estimated variance-covariance matrix of the estimated regression coefficients when R is nonsingular and is from an unrestricted regression fit. (Output)
See Comments for an explanation of COVB when R is singular or R is from a restricted regression fit. If R is not needed, COVB and R can share the same storage locations.
Optional Arguments
NCOEF — Number of regression coefficients in the model. (Input) Default: NCOEF = size (R,1).
LDR — Leading dimension of R exactly as specified in the dimension statement in the calling program. (Input)
Default: LDR = size (R,1).
LDCOVB — Leading dimension of COVB exactly as specified in the dimension statement in the calling program. (Input)
Default: LDCOVB = size (COVB,1).
FORTRAN 90 Interface
Generic: CALL RCOVB(R, S2, COVB [,…])
Specific: The specific interface names are S_RCOVB and D_RCOVB. FORTRAN 77 Interface
Single: CALL RCOVB (NCOEF, R, LDR, S2, COVB, LDCOVB) Double: The double precision name is DRCOVB.
Example 1
This example uses a data set discussed by Draper and Smith (1981, pages 629-630). This data set is put into the matrix X by routine GDATA (see Chapter 19, Utilities). There are 4 independent variables and 1 dependent variable. Routine RGIVN (page 117) is invoked to fit the regression model, and RCOVB is invoked to compute summary statistics.
! SPECIFICATIONS FOR LOCAL VARIABLES USE RCOVB_INT
USE GDATA_INT USE RGIVN_INT USE WRRRL_INT
INTEGER INTCEP, LDB, LDCOEF, LDCOVB, LDR, LDSCPE, LDX, NCOEF, &
NDEP, NDX, NIND
PARAMETER (INTCEP=1, LDX=13, NDEP=1, NDX=5, NIND=4, &
166 · Chapter 2: Regression IMSL STAT/LIBRARY LDSCPE=NDEP, NCOEF=INTCEP+NIND, LDB=NCOEF, &
LDCOEF=NCOEF, LDCOVB=NCOEF, LDR=NCOEF)
!
INTEGER IDEP, IDO, IFRQ, IIND, INDDEP(1), INDIND(1), IRANK, &
ICEN, IWT, NCOL, NRMISS, NROW
REAL B(LDB,NDEP), COVB(LDCOVB,5), DFE, R(LDR,NCOEF), &
S2, SCPE(LDSCPE,NDEP), X(LDX,NDX) CHARACTER CLABEL(6)*10, RLABEL(5)*10
!
DATA RLABEL/’Intercept’, ’X1’, ’X2’, ’X3’, ’X4’/
DATA CLABEL/’ ’, ’Intercept’, ’X1’, ’X2’, ’X3’, ’X4’/
!
CALL GDATA (5, X, NROW, NCOL) IIND = -NIND
IDEP = -NDEP
CALL RGIVN (X, IIND, INDIND, IDEP, INDDEP, B, R=R, DFE=DFE, &
SCPE=SCPE) S2 = SCPE(1,1)/DFE
!
CALL RCOVB (R, S2, COVB)
CALL WRRRL (’COVB’, COVB, RLABEL, CLABEL, FMT=’(2W10.4)’)
!
END
Output
COVB
Intercept X1 X2 X3 X4 Intercept 4910.0 -50.51 -50.60 -51.66 -49.60 X1 -50.5 0.55 0.51 0.55 0.51 X2 -50.6 0.51 0.52 0.53 0.51 X3 -51.7 0.55 0.53 0.57 0.52 X4 -49.6 0.51 0.51 0.52 0.50
Comments
When R is nonsingular and comes from an unrestricted regression fit, COVB is the estimated variance-covariance matrix of the estimated regression coefficients, and COVB = s2(RTR)-1.
Otherwise, variances and covariances of estimable functions of the regression coefficients can be obtained using COVB, and COVB = s2GDGT. Here, D is the diagonal matrix with diagonal elements equal to 0 if the corresponding rows of R are restrictions and with diagonal elements equal to one otherwise. Also, G is a particular generalized inverse of R. See the Algorithm section.
Description
Routine RCOVB computes an estimated variance-covariance matrix of estimated regression parameters from the R matrix in several models. In the simplest situation, the model is a general linear model given by y = Xb + e where y is the n ´ 1 vector of responses, X is the n ´ p matrix of regressors, b is the p ´ 1 vector of regression coefficients, and e is the n ´ 1 vector of errors whose elements are each independently distributed with mean 0 and variance s2. Routine RGIVN (page 117) can be used to get the fit of the model and the R matrix.
IMSL STAT/LIBRARY Chapter 2: Regression · 167 If the i-th element of e has variance s2/wi and the weights wi are used in the fit of the model, RCOVB produces the estimated variance-covariance matrix from the R matrix in the weighted least squares fit. More generally, if the variance-covariance matrix of e is s2V, RCOVB can be used to produce the estimated variance-covariance matrix from the generalized least-squares fit.
(Routine RGIVN can be used to perform a generalized least-squares fit, by regressing y* on X* where y* = (T-1)Ty, X* = (T-1)TX and T satisfies TTT = V.)
If the general linear model has the restriction Hb = g on the regression parameters and this restriction is used in the fit of the model by routine RLEQU (page 142), RCOVB produces the estimated variance-covariance from the R matrix in the restricted least squares fit.
Routine RCOVB computes an estimated variance-covariance matrix for the estimated regression coefficients,
Bˆ
in a fitted multivariate general linear model. The model is Y = XB + E where Y is the n ´ q matrix of responses, X is the n ´ p matrix of regressors, B is the p ´ q matrix of regression coefficients, and E is the n ´ q matrix of errors whose rows are each independently distributed as a q-dimensional multivariate normal each with mean vector 0 and variance-covariance matrix S. Let
(
ˆ ˆ1 2 ˆ)
ˆ , , , q
B= b b K b The estimated covariance matrix
(
ˆ ˆ) ( )
1Cov b bi, j =s X Xij T - Here, sij (input in S2) is the estimate of the ij-th element of S.
If a nonlinear regression model is fit using routine RNLIN (page 304), RCOVB produces the asymptotic estimated variance-covariance matrix from the R matrix in that fit.
If R is singular, corresponding to rank(R) < p, a generalized inverse is used to compute COVB. For a matrix G to be a gi(i = 1, 2, 3, or 4) inverse of a matrix A, G must satisfy conditions j(for j
£ i) for the Moore-Penrose inverse but, generally, must fail conditions k (for k > i). The four conditions for G to be a Moore-Penrose inverse of A are as follows:
1. AGA = A 2. GAG = G 3. AG is symmetric 4. GA is symmetric
In the case that R is singular, the method for obtaining COVB follows the discussion of Maindonald (1984, pages 101-103). Let Z be the diagonal matrix with diagonal elements defined by
168 · Chapter 2: Regression IMSL STAT/LIBRARY if 0
1
if 0
0
ii ii
ii
z r
r ì ¹
= íî =
Let G be the solution to RG = Z obtained by setting the i-th ({i : rii= 0}) row of G to zero. COVB is set to s2GGT. (G is a g3 inverse of R. For any g3 inverse of R, represented by
g3
R the result
3 3
g g T
R R
is a symmetric g2 inverse of RTR = XTX. See Sallas and Lionti [1988].)
Note that COVB can only be used to get variances and covariances of estimable functions of the regression coefficients, i.e., nonestimable functions (linear combinations of the regression coefficients not in the space spanned by the nonzero rows of R) must not be used. See, for example, Maindonald (1984, pages 166-168) for a discussion of estimable functions.
The preceding discussion can be modified to include the restricted least-squares problem. The modification is based on the work of Stirling (1981). Let the matrix D = diag(d1, d2, ¼, dp) be a diagonal matrix with elements dii = 0 if the i-th row of R corresponds to a restriction and 1 otherwise. In the unrestricted case, D is simply the p ´ p identity matrix. The formula for COVB is s2GDGT.
Additional Example Example 2
In this example, routine RNLIN (page 304) is first invoked to fit the following nonlinear regression model discussed by Neter, Wasserman, and Kutner (1983, pages 475-478):
1 2xi 1, 2, , 15
i i
y =qeq +e i= K
Then, RCOVB is used to compute the estimated asymptotic variance-covariance matrix of the estimated nonlinear regression parameters. Finally, the diagonal elements of the output matrix from RCOVB are used together with routine TIN (see Chapter 17, Probability Distribution Functions and Inverses) to compute 95% confidence intervals on the regression parameters.
USE IMSL_LIBRARIES
INTEGER LDR, NOBS, NPARM
PARAMETER (NOBS=15, NPARM=2, LDR=NPARM)
!
INTEGER I, IDERIV, IRANK, ISETNG, NOUT
REAL A, DFE, R(LDR,NPARM), SQRT, SSE, THETA(NPARM) INTRINSIC SQRT
EXTERNAL EXAMPL
!
DATA THETA/60.0, -0.03/
!
CALL UMACH (2, NOUT)
IMSL STAT/LIBRARY Chapter 2: Regression · 169
!
IDERIV = 1
CALL RNLIN (EXAMPL, THETA, IDERIV=IDERIV, R=R, DFE=DFE, SSE=SSE)
!
CALL RCOVB (R, SSE/DFE, R)
! Print ISETNG=2
CALL WROPT (-6, ISETNG, 0)
CALL WRRRN (’Estimated Asymptotic Variance-Covariance Matrix’, &
R)
! Compute and print 95 percent
! confidence intervals.
WRITE (NOUT,*)
WRITE (NOUT,*) ’ 95% Confidence Intervals ’ WRITE (NOUT,*) ’ Estimate Lower Limit Upper Limit’
DO 10 I=1, NPARM
A = TIN(0.975,DFE)*SQRT(R(I,I))
WRITE (NOUT,’(1X, F10.3, 2F13.3)’) THETA(I), THETA(I) - A, &
THETA(I) + A 10 CONTINUE
END
!
SUBROUTINE EXAMPL (NPARM, THETA, IOPT, IOBS, FRQ, WT, E, DE, &
IEND)
INTEGER NPARM, IOPT, IOBS, IEND
REAL THETA(NPARM), FRQ, WT, E, DE(NPARM)
!
INTEGER NOBS PARAMETER (NOBS=15)
!
REAL EXP, XDATA(NOBS), YDATA(NOBS) INTRINSIC EXP
!
DATA YDATA/54.0, 50.0, 45.0, 37.0, 35.0, 25.0, 20.0, 16.0, 18.0, &
13.0, 8.0, 11.0, 8.0, 4.0, 6.0/
DATA XDATA/2.0, 5.0, 7.0, 10.0, 14.0, 19.0, 26.0, 31.0, 34.0, &
38.0, 45.0, 52.0, 53.0, 60.0, 65.0/
!
IF (IOBS .LE. NOBS) THEN WT = 1.0E0
FRQ = 1.0E0 IEND = 0
IF (IOPT .EQ. 0) THEN
E = YDATA(IOBS) - THETA(1)*EXP(THETA(2)*XDATA(IOBS)) ELSE
DE(1) = -EXP(THETA(2)*XDATA(IOBS))
DE(2) = -THETA(1)*XDATA(IOBS)*EXP(THETA(2)*XDATA(IOBS)) END IF
ELSE
IEND = 1 END IF RETURN END
170 · Chapter 2: Regression IMSL STAT/LIBRARY
Output
Estimated Asymptotic Variance-Covariance Matrix 1 2
1 2.16701E+00 -1.78121E-03 2 -1.78121E-03 2.92786E-06 95% Confidence Intervals Estimate Lower Limit Upper Limit 58.603 55.423 61.784 -0.040 -0.043 -0.036