Chapter Contents |
Previous |
Next |
The AUTOREG Procedure |
The regression model with autocorrelated disturbances is as follows:
In these equations, y_{t} are the dependent values, x_{t} is a column vector of regressor variables, is a column vector of structural parameters, and is normally and independently distributed with a mean of 0 and a variance of ^{2}. Note that in this parameterization, the signs of the autoregressive parameters are reversed from the parameterization documented in most of the literature.
PROC AUTOREG offers four estimation methods for the autoregressive error model. The default method, Yule-Walker (YW) estimation, is the fastest computationally. The Yule-Walker method used by PROC AUTOREG is described in Gallant and Goebel (1976). Harvey (1981) calls this method the two-step full transform method. The other methods are iterated YW, unconditional least squares (ULS), and maximum likelihood (ML). The ULS method is also referred to as nonlinear least squares (NLS) or exact least squares (ELS).
You can use all of the methods with data containing missing values, but you should use ML estimation if the missing values are plentiful. See the section "Alternative Autocorrelation Correction Methods" later in this chapter for further discussion of the advantages of different methods.
and let the variance matrix of the error vector be
If the vector of autoregressive parameters is known, the matrix V can be computed from the autoregressive parameters. is then . Given , the efficient estimates of regression parameters can be computed using generalized least squares (GLS). The GLS estimates then yield the unbiased estimate of the variance ^{2},
The Yule-Walker method alternates estimation of using generalized least squares with estimation of using the Yule-Walker equations applied to the sample autocorrelation function. The YW method starts by forming the OLS estimate of . Next, is estimated from the sample autocorrelation function of the OLS residuals using the Yule-Walker equations. Then V is estimated from the estimate of , and is estimated from V and the OLS estimate of . The autocorrelation corrected estimates of the regression parameters are then computed by GLS using the estimated matrix. These are the Yule-Walker estimates.
If the ITER option is specified, the Yule-Walker residuals are used to form a new sample autocorrelation function, the new autocorrelation function is used to form a new estimate of and V, and the GLS estimates are recomputed using the new variance matrix. This alternation of estimates continues until either the maximum change in the estimate between iterations is less than the value specified by the CONVERGE= option or the maximum number of allowed iterations is reached. This produces the Iterated Yule-Walker estimates. Iteration of the estimates may not yield much improvement.
The Yule-Walker equations, solved to obtain and a preliminary estimate of , are
Here r = (r_{1}, ... , r_{m})', where r_{i} is the lag i sample autocorrelation. The matrix R is the Toeplitz matrix whose i,jth element is r_{|i-j|}. If you specify a subset model, then only the rows and columns of R and r corresponding to the subset of lags specified are used.
If the BACKSTEP option is specified, for purposes of significance testing, the matrix [R r] is treated as a sum-of-squares-and-crossproducts matrix arising from a simple regression with N-k observations, where k is the number of estimated parameters.
The unconditional sum of squares for the model, S, is
The full log likelihood function for the autoregressive error model is
where denotes determinant of V. For the ML method, the likelihood function is maximized by minimizing an equivalent sum-of-squares function.
Maximizing l with respect to (and concentrating out of the likelihood) and dropping the constant term produces the concentrated log likelihood function
Rewriting the variable term within the logarithm gives
PROC AUTOREG computes the ML estimates by minimizing the objective function .
The maximum likelihood estimates may not exist for some data sets (Anderson and Mentz 1980). This is the case for very regular data sets, such as an exact linear trend.
If there are no missing values, then l+j = N, the number of observations. In this case, the Toeplitz matrix of autocorrelations, R, is at least positive semidefinite. If there are missing values, these autocorrelation estimates of r can yield an R matrix that is not positive semidefinite. If such estimates occur, a warning message is printed, and the estimates are tapered by exponentially declining weights until R is positive definite.
In all of the estimation methods, the original data are transformed by the inverse of the Cholesky root of V. Let L denote the Cholesky root of V, that is V = LL' with L lower triangular. For an AR(m) model, L^{-1} is a band diagonal matrix with m anomalous rows at the beginning and the autoregressive parameters along the remaining rows. Thus, if there are no missing values, after the first m-1 observations the data are transformed as
The transformation is carried out using a Kalman filter, and the lower triangular matrix L is never directly computed. The Kalman filter algorithm, as it applies here, is described in Harvey and Phillips (1979) and Jones (1980). Although L is not computed explicitly, for ease of presentation the remaining discussion is in terms of L. If there are missing values, then the submatrix of L consisting of the rows and columns with nonmissing values is used to generate the transformations.
The Gauss-Newton algorithm requires the derivatives of e or with respect to the parameters. The derivatives with respect to the parameter vector are
These derivatives are computed by the transformation described previously. The derivatives with respect to are computed by differentiating the Kalman filter recurrences and the equations for the initial conditions.
The variance-covariance matrix for the components of b is taken as s^{2}(X' V^{-1} X)^{-1} for the Yule-Walker method. For the ULS and ML methods, the variance-covariance matrix of the parameter estimates is computed as s^{2} (J'J)^{-1}. For the ULS method, J is the matrix of derivatives of e with respect to the parameters. For the ML method, J is the matrix of derivatives of divided by . The estimate of the variance-covariance matrix of b assuming that is known is s^{2} (X' V^{-1}X)^{-1}.
Park and Mitchell (1980) investigated the small sample performance of the standard error estimates obtained from some of these methods. In particular, simulating an AR(1) model for the noise term, they found that the standard errors calculated using GLS with an estimated autoregressive parameter underestimated the true standard errors. These estimates of standard errors are the ones calculated by PROC AUTOREG with the Yule-Walker method.
The estimates of the standard errors calculated with the ULS or ML methods take into account the joint estimation of the AR and the regression parameters and may give more accurate standard-error values than the YW method. At the same values of the autoregressive parameters, the ULS and ML standard errors will always be larger than those computed from Yule-Walker. However, simulations of the models used by Park and Mitchell suggest that the ULS and ML standard error estimates can also be underestimates. Caution is advised, especially when the estimated autocorrelation is high and the sample size is small.
High autocorrelation in the residuals is a symptom of lack of fit. An autoregressive error model should not be used as a nostrum for models that simply do not fit. It is often the case that time series variables tend to move as a random walk. This means that an AR(1) process with a parameter near one absorbs a great deal of the variation. See Example 8.3 later in this chapter, which fits a linear trend to a sine wave.
For ULS or ML estimation, the joint variance-covariance matrix of all the regression and autoregression parameters is computed. For the Yule-Walker method, the variance-covariance matrix is computed only for the regression parameters.
Chapter Contents |
Previous |
Next |
Top |
Copyright © 1999 by SAS Institute Inc., Cary, NC, USA. All rights reserved.