Autoregressive Processes

4.3 Autoregressive Processes 67

(4.3.4)

Setting k= 1, we get . With k= 2, we ob tain

. Now it is easy to see that in general

(4.3.5) and thus

(4.3.6) Since , the magnitude of the autocorrelation function decreases exponentially as the number of lags, k, increases. If , all correlations are positive; if , the lag 1 autocorrelation is negative (ρ₁=φ) and the signs of successive autocorrelations alternate from positive to negative, with their magnitudes decreasing exponentially. Portions of the graphs of several autocorrelation functions are displayed in Exhibit 4.12.

Exhibit 4.12 Autocorrelation Functions for Several AR(1) Models

Notice that for φ near , the exponential decay is quite slow (for example, (0.9)⁶ = 0.53), but for smaller φ, the decay is quite rapid (for example, (0.4)⁶ = 0.00410). With φ near , the strong correlation will extend over many lags and produce a relatively

γ_k = φγ_k–1 for k = 1, 2, 3,...

γ₁ = φγ₀ = φσ_e²⁄(1–φ²) γ₂ = φ²σ_e²⁄(1–φ²)

γ_k φ^k σ_e² 1–φ²

---=

ρ_k γ_k γ₀ --- φ^k

= = for k = 1, 2, 3,...

φ <1

0< <φ 1 1< <φ 0

–

2 4 6 8 10 12

−1.00.01.0

Lag ρρk

●

●

●

●

●

●

●

●

●

●

●

●

2 4 6 8 10 12

−1.00.01.0

Lag ρρk

●

●

●

● ● ● ● ● ● ● ● ●

2 4 6 8 10 12

0.00.40.8

Lag ρρk

●

●

●

●

●

● ●

● ●

● ● ●

2 4 6 8 10 12

0.00.40.8

Lag ρρk

●

●

● ● ● ● ● ● ● ● ● ●

φ = 0.9 φ = 0.4

φ = −0.8 φ = −0.5

±1

±1

smooth series if φ is positive and a very jagged series if φ is negative.

Exhibit 4.13 displays the time plot of a simulated AR(1) process with φ = 0.9.

Notice how infrequently the series crosses its theoretical mean of zero. There is a lot of inertia in the series—it hangs together, remaining on the same side of the mean for extended periods. An observer might claim that the series has several trends. We know that in fact the theoretical mean is zero for all time points. The illusion of trends is due to the strong autocorrelation of neighboring values of the series.

Exhibit 4.13 Time Plot of an AR(1) Series with φ = 0.9

> win.graph(width=4.875, height=3,pointsize=8)

> data(ar1.s); plot(ar1.s,ylab=expression(Y[t]),type='o')

The smoothness of the series and the strong autocorrelation at lag 1 are depicted in the lag plot shown in Exhibit 4.14.

● ● ●

● ● ●

●●

●

●

● ●

●

●

●

●

●

●

●

● ●●

● ●

●

●● ●

●

●

● ● ●● ● ●

●

● ● ●

●

●● ●

●

●

●●

●

● ● ●

●

●

●●● ●

●

●

Time Yt

0 10 20 30 40 50 60

−2024

4.3 Autoregressive Processes 69

Exhibit 4.14 Plot of Y_t versus Y_t−1 for AR(1) Series of Exhibit 4.13

> win.graph(width=3, height=3,pointsize=8)

> plot(y=ar1.s,x=zlag(ar1.s),ylab=expression(Y[t]), xlab=expression(Y[t-1]),type='p')

This AR(1) model also has strong positive autocorrelation at lag 2, namely ρ₂ = (0.9)² = 0.81. Exhibit 4.15 shows this quite well.

Exhibit 4.15 Plot of Y_t versus Y_t−2 for AR(1) Series of Exhibit 4.13

> plot(y=ar1.s,x=zlag(ar1.s,2),ylab=expression(Y[t]), xlab=expression(Y[t-2]),type='p')

●●

● ●●

● ●

●

●

● ●

●

●

●

●

●

●

●

●● ●

● ●

●

● ● ●

●

●

● ●●

● ●●

●

● ●●

●

●

● ●

●

●

● ●

●

● ● ●

●

●

● ●

● ●

●

●

−2 0 2 4

−2024

Y_t−1 Yt

●

●● ●

●●

●

●

● ●

●

●

●

●

●

●

●

● ● ●

● ●

●

● ● ●

●

●

● ● ●

● ●●

●

● ● ●

●

●

● ●

●

●

● ●

●

● ● ●

●

●

● ●

●●

●

●

−2 0 2 4

−2024

Y_t−2 Yt

Finally, at lag 3, the autocorrelation is still quite high: ρ₃ = (0.9)³ = 0.729. Exhibit 4.16 confirms this for this particular series.

Exhibit 4.16 Plot of Y_t versus Y_t−3 for AR(1) Series of Exhibit 4.13

> plot(y=ar1.s,x=zlag(ar1.s,3),ylab=expression(Y[t]), xlab=expression(Y[t-3]),type='p')

The General Linear Process Version of the AR(1) Model

The recursive definition of the AR(1) process given in Equation (4.3.2) is extremely useful for interpretating the model. For other purposes, it is convenient to express the AR(1) model as a general linear process as in Equation (4.1.1). The recursive definition is valid for all t. If we use this equation with t replaced by t−1, we get

. Substituting this into the original expression gives

If we repeat this substitution into the past, say k − 1 times, we get

(4.3.7) Assuming and letting k increase without bound, it seems reasonable (this is almost a rigorous proof) that we should obtain the infinite series representation

(4.3.8)

●●●

●●

●

●

● ●

●

●

●

●

●

●

●

● ●●

●●

●

●●

●

●

●

● ●●

●●●

●

●●

●

●

●

● ●

●

●

● ●

●

●●

●

●

●

● ●

● ●

●

●

−2 0 2 4

−2024

Yt−3

Yt

Y_t–1 = φY_t–2+e_t–1

Y_t = φ φY( _t–2+e_t–1)+e_t e_t+φe_t–1+φ²Y_t–2

=

Y_t = e_t+φe_t–1+φ²e_t–2+… φ+ ^k–1e_t–k+1+φ^kY_t–k φ <1

Y_t = e_t+φe_t–1+φ²e_t–2+φ³e_t–3+…

4.3 Autoregressive Processes 71 This is in the form of the general linear process of Equation (4.1.1) with , which we already investigated in Section 4.1 on page 55. Note that this representation reemphasizes the need for the restriction .

Stationarity of an AR(1) Process

It can be shown that, subject to the restriction that e_t be independent of Y_t−1, Y_t−2, Y_t−3,… and that , the solution of the AR(1) defining recursion

will be stationary if and only if . The requirement is usually called the stationarity condition for the AR(1) process (See Box, Jenkins, and Reinsel, 1994, p. 54; Nelson, 1973, p. 39; and Wei, 2005, p. 32) even though more than stationarity is involved. See especially Exercises 4.16, 4.18, and 4.25.

At this point, we should note that the autocorrelation function for the AR(1) process has been derived in two different ways. The first method used the general linear process representation leading up to Equation (4.1.3). The second method used the defining recursion and the development of Equations (4.3.4), (4.3.5), and (4.3.6). A third derivation is obtained by multiplying both sides of Equation (4.3.7) by Y_t−k, taking expected values of both sides, and using the fact that e_t, e_t−1, e_t−2, ... , e_{t− (k−1)} are independent of Y_t−k. The second method should be especially noted since it will generalize nicely to higher-order processes.

The Second-Order Autoregressive Process Now consider the series satisfying

(4.3.9) where, as usual, we assume that e_t is independent of Y_t−1, Y_t−2, Y_t−3, ... . To discuss stationarity, we introduce the AR characteristic polynomial

and the corresponding AR characteristic equation

We recall that a quadratic equation always has two roots (possibly complex).

Stationarity of the AR(2) Process

It may be shown that, subject to the condition that e_t is independent of Y_t−1, Y_t−2, Y_t−3,..., a stationary solution to Equation (4.3.9) exists if and only if the roots of the AR characteristic equation exceed 1 in absolute value (modulus). We sometimes say that the roots should lie outside the unit circle in the complex plane. This statement will general-ize to the pth-order case without change.^†

† It also applies in the first-order case, where the AR characteristic equation is just = 0 with root 1/φ, which exceeds 1 in absolute value if and only if .

ψ_j = φ^j φ <1

σ_e²>0 Y_t = φY_t–1+e_t

φ <1 φ <1

Y_t = φY_t–1+e_t

Y_t = φ₁Y_t–1+φ₂Y_t–2+e_t

φ( )x = 1–φ₁x–φ₂x²

1–φ₁x–φ₂x² = 0

1–φx φ <1

In the second-order case, the roots of the quadratic characteristic equation are easily found to be

(4.3.10) For stationarity, we require that these roots exceed 1 in absolute value. In Appendix B, page 84, we show that this will be true if and only if three conditions are satisfied:

(4.3.11) As with the AR(1) model, we call these the stationarity conditions for the AR(2) model. This stationarity region is displayed in Exhibit 4.17.

Exhibit 4.17 Stationarity Parameter Region for AR(2) Process

The Autocorrelation Function for the AR(2) Process

To derive the autocorrelation function for the AR(2) case, we take the defining recursive relationship of Equation (4.3.9), multiply both sides by Y_t−k, and take expectations.

Assuming stationarity, zero means, and that e_t is independent of Y_t−k, we get

(4.3.12) or, dividing through by γ₀,

(4.3.13) Equations (4.3.12) and/or (4.3.13) are usually called the Yule-Walker equations, espe-cially the set of two equations obtained for k = 1 and 2. Setting k = 1 and using ρ₀ = 1 and ρ₋₁ = ρ₁, we get and so

φ₁± φ₁²+4φ₂ 2φ₂ ---–

φ₁+φ₂<1, φ₂–φ₁<1, and φ₂ <1

−2 −1 0 1 2

−1.0−0.50.00.51.0

φφ1

φφ2

real roots

complex roots

φ₁²+4φ₂ = 0

γ_k = φ₁γ_k–1+φ₂γ_k–2 for k = 1, 2, 3, ...

ρ_k = φ₁ρ_k–1+φ₂ρ_k–2 for k = 1, 2, 3, ...

ρ₁ = φ₁+φ₂ρ₁

4.3 Autoregressive Processes 73

(4.3.14) Using the now known values for ρ₁ (and ρ₀), Equation (4.3.13) can be used with k = 2 to obtain

(4.3.15)

Successive values of ρ_k may be easily calculated numerically from the recursive rela-tionship of Equation (4.3.13).

Although Equation (4.3.13) is very efficient for calculating autocorrelation values numerically from given values of φ₁ and φ₂, for other purposes it is desirable to have a more explicit formula for ρ_k. The form of the explicit solution depends critically on the roots of the characteristic equation . Denoting the reciprocals of these roots by G₁ and G₂, it is shown in Appendix B, page 84, that

For the case G₁ ≠ G₂, it can be shown that we have

(4.3.16) If the roots are complex (that is, if ), then ρ_k may be rewritten as

(4.3.17)

where and Θ and Φ are defined by and

.

For completeness, we note that if the roots are equal ( ), then we have (4.3.18) A good discussion of the derivations of these formulas can be found in Fuller (1996, Section 2.5).

The specific details of these formulas are of little importance to us. We need only note that the autocorrelation function can assume a wide variety of shapes. In all cases, the magnitude of ρ_k dies out exponentially fast as the lag k increases. In the case of com-plex roots, ρ_k displays a damped sine wave behavior with damping factor R, , frequency Θ, and phase Φ. Illustrations of the possible shapes are given in Exhibit 4.18. (The R function ARMAacf discussed on page 450 is useful for plotting.)

ρ₁ φ₁ 1–φ₂

---=

ρ₂ = φ₁ρ₁+φ₂ρ₀ φ₂(1–φ₂) φ+ ₁²

1–φ₂

---=

1–φ₁x–φ₂x² = 0

G₁ φ₁– φ₁²+4φ₂ ---2

= and G₂ φ₁+ φ₁²+4φ₂

---2

=

ρ_k (1–G₂²)G₁^k+1–(1–G₁²)G₂^k+1 G₁–G₂

( )(1+G₁G₂)

---= for k≥0

φ₁²+4φ₂<0 ρ_k R^ksin(Θk+Φ)

Φ sin( )

---= for k≥0

R = –φ₂ cos( )Θ = φ₁⁄(2 –φ₂) tan( )Φ = 1–φ₂

( )⁄(1+φ₂)

[ ]

φ₁²+4φ₂ = 0 ρ_k 1 1+φ₂

1–φ₂ ---k

⎝ + ⎠

⎛ ⎞ φ¹

---2

⎝ ⎠⎛ ⎞^k

= for k = 0, 1, 2,...

0≤R<1

Exhibit 4.18 Autocorrelation Functions for Several AR(2) Models

Exhibit 4.19 displays the time plot of a simulated AR(2) series with φ₁= 1.5 and φ₂=−0.75. The periodic behavior of ρ_k shown in Exhibit 4.18 is clearly reflected in the nearly periodic behavior of the series with the same period of 360/30 = 12 time units. If Θ is measured in radians, 2π/Θ is sometimes called the quasi-period of the AR(2) pro-cess.

Exhibit 4.19 Time Plot of an AR(2) Series with φ₁ = 1.5 and φ₂ = −0.75

> win.graph(width=4.875,height=3,pointsize=8)

> data(ar2.s); plot(ar2.s,ylab=expression(Y[t]),type='o')

2 4 6 8 10 12

−0.50.00.51.0

Lag ρρk

●

●

●

●

● ● ●

●

●

● ● ●

2 4 6 8 10 12

−0.50.00.51.0

Lag ρρk

●

●

● ●

●

● ● ●

● ● ● ●

φ₁ = 1.5, φ₂ = −0.75 φ₁ = 1.0, φ₂ = −0.6

2 4 6 8 10 12

0.00.40.8

Lag ρρk

●

●

●

●

● ●

● ● ● ● ● ●

2 4 6 8 10 12

0.00.40.8

Lag ρρk

●

●

●

●

● ● ● ● ● ● ● ●

φ₁ = 1.0, φ₂ = −0.25 φ₁ = 0.5, φ₂ = 0.25

●●

●

●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●●

●

●●●

●

●●●

●

●

●

●●

●●

●

●

●

●●●

●

●

●

●●

●●

●●●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●●●●●

●

●

●●●●

●●

●

●

●●

●●●

●

●●

●●

●

●

●

●

●

●

Time Yt

0 20 40 60 80 100 120

−4−20246

4.3 Autoregressive Processes 75

The Variance for the AR(2) Model

The process variance γ₀ can be expressed in terms of the model parameters φ₁, φ₂, and as follows: Taking the variance of both sides of Equation (4.3.9) yields

(4.3.19) Setting k= 1 in Equation (4.3.12) gives a second linear equation for γ₀ and γ₁, , which can be solved simultaneously with Equation (4.3.19) to obtain

(4.3.20)

The ψ-Coefficients for the AR(2) Model

The ψ-coefficients in the general linear process representation for an AR(2) series are more complex than for the AR(1) case. However, we can substitute the general linear process representation using Equation (4.1.1) for Y_t, for Y_t−1, and for Y_t−2 into . If we then equate coefficients of e_j, we get the recursive relationships

(4.3.21)

These may be solved recursively to obtain ψ₀= 1, ψ₁=φ₁, , and so on.

These relationships provide excellent numerical solutions for the ψ-coefficients for given numerical values of φ₁ and φ₂.

One can also show that, for G₁ ≠ G₂, an explicit solution is

(4.3.22) where, as before, G₁ and G₂ are the reciprocals of the roots of the AR characteristic equation. If the roots are complex, Equation (4.3.22) may be rewritten as

(4.3.23) a damped sine wave with the same damping factor R and frequency Θ as in Equation (4.3.17) for the autocorrelation function.

For completeness, we note that if the roots are equal, then

(4.3.24) σ_e²

γ₀ = (φ₁²+φ₂²)γ₀+2φ₁φ₂γ₁+σ_e² γ₁ = φ₁γ₀+φ₂γ₁

γ₀ (1–φ₂)σ_e² 1–φ₂

( )(1–φ₁²–φ₂²)–2φ₂φ₁²

---=

1–φ₂ 1+φ₂

---⎝ ⎠

⎛ ⎞ σ_e²

1–φ₂ ( )²–φ₁²

---=

Y_t = φ₁Y_t–1+φ₂Y_t–2+e_t

ψ₀ = 1 ψ₁–φ₁ψ₀ = 0

ψ_j–φ₁ψ_j–1–φ₂ψ_j–2 = 0 for j = 2, 3, ...⎭⎪⎬⎪⎫

ψ₂ = φ₁²+φ₂

ψ_j G₁^j+1–G₂^j+1 G₁–G₂

---=

ψ_j R^j sin[(j+1)Θ] Θ sin( )

---⎩ ⎭

⎨ ⎬

⎧ ⎫

=

ψ_j = (1+j)φ₁^j

The General Autoregressive Process Consider now the pth-order autoregressive model

(4.3.25) with AR characteristic polynomial

(4.3.26) and corresponding AR characteristic equation

(4.3.27) As noted earlier, assuming that e_t is independent of Y_t−1, Y_t−2, Y_t−3,... a station-ary solution to Equation (4.3.27) exists if and only if the p roots of the AR characteristic equation each exceed 1 in absolute value (modulus). Other relationships between poly-nomial roots and coefficients may be used to show that the following two inequalities are necessary for stationarity. That is, for the roots to be greater than 1 in modulus, it is necessary, but not sufficient, that both

(4.3.28) Assuming stationarity and zero means, we may multiply Equation (4.3.25) by Y_t−k, take expectations, divide by γ₀, and obtain the important recursive relationship

(4.3.29) Putting k= 1, 2,..., and p into Equation (4.3.29) and using ρ₀= 1 and ρ_−k=ρ_k, we get the general Yule-Walker equations

(4.3.30)

Given numerical values for φ₁, φ₂, ... , φ_p, these linear equations can be solved to obtain numerical values for ρ₁, ρ₂,..., ρ_p. Then Equation (4.3.29) can be used to obtain numerical values for ρ_k at any number of higher lags.

Noting that

we may multiply Equation (4.3.25) by Y_t, take expectations, and find Y_t = φ₁Y_t–1+φ₂Y_t–2+… φ+ _pY_t–p+e_t

φ( )x = 1–φ₁x–φ₂x²–…–φ_px^p

1–φ₁x–φ₂x²–…–φ_px^p = 0

φ₁+φ₂+… φ+ _p<1 and φ_p <1 ⎭⎬⎫

ρ_k = φ₁ρ_k–1+φ₂ρ_k–2+φ₃ρ_k–3+… φ+ _pρ_k–p for k≥1

ρ₁ = φ₁+φ₂ρ₁+φ₃ρ₂+… φ+ _pρ_p–1 ρ₂ = φ₁ρ₁+φ₂+φ₃ρ₁+… φ+ _pρ_p–2

...

ρ_p = φ₁ρ_p–1+φ₂ρ_p–2+φ₃ρ_p–3+… φ+ _p⎭⎪⎪⎬⎪⎪⎫

E e( _tY_t) = E e[ _t(φ₁Y_t–1+φ₂Y_t–2+… φ+ _pY_t–p+e_t)] = E e( )_t² = σ_e²

γ₀ = φ₁γ₁+φ₂γ₂+… φ+ _pγ_p+σ_e²

No documento Statistics Texts in Statistics Series Editors: G. Casella S. Fienberg I. Olkin (páginas 79-90)