Chapter Contents |
Previous |
Next |

The NLIN Procedure |

For the system of equations represented by the nonlinear model

The second method is to solve the nonlinear "normal" equations

The iterative techniques that PROC NLIN uses are similar to a series
of linear regressions involving the matrix **X**
evaluated for the current values of and
, the residuals
evaluated for the current values of .

The iterative process begins at some point .Then **X** and **Y** are used
to compute a such that

The Gauss-Newton and Marquardt iterative methods regress the residuals onto the partial derivatives of the model with respect to the parameters until the estimates converge. The Newton iterative method regresses the residuals onto a function of the first and second derivatives of the model with respect to the parameters until the estimates converge. Analytical first- and second-order derivatives are automatically computed.

If the automatic variables _WEIGHT_ and _RESID_ are used, then

**W**^{SSE}- is an
*n*×*n*diagonal matrix with elements*w*_{i}^{SSE}of weights from the _WEIGHT_ variable. Each element*w*_{i}^{SSE}contains the value of _WEIGHT_ for the*i*th observation.

**r**- is a vector with elements
*r*_{i}from _RESID_. Each element*r*_{i}contains the value of _RESID_ evaluated for the*i*th observation.

Using the method of steepest descent, let

**W**^{SSE}- is an
*n*×*n*diagonal matrix with elements*w*_{i}^{SSE}of weights from the _WEIGHT_ variable. Each element*w*_{i}^{SSE}contains the value of _WEIGHT_ for the*i*th observation. **W**^{XPX}- is an
*n*×*n*diagonal matrix with elements*w*_{i}^{XPX}of weights from the _WGTJPJ_ variable.

Each element*w*_{i}^{XPX}contains the value of _WGTJPJ_ for the*i*th observation. **r**- is a vector with elements
*r*_{i}from the _RESID_ variable. Each element*r*_{i}contains the value of _RESID_ evaluated for the*i*th observation.

Substituting the first two terms of this series into the normal equations

Marquardt's studies indicate that the average angle between Gauss-Newton and steepest descent directions is about . A choice of between 0 and infinity produces a compromise direction.

By default, PROC NLIN chooses to start and computes a . If SSE, then for the next iteration. Each time SSE, then .

Note: If the SSE decreases on each iteration, then , and you are essentially using the Gauss-Newton method. If SSE does not improve, then is increased until you are moving in the steepest descent direction.

Marquardt's method is equivalent to performing a series of ridge regressions and is useful when the parameter estimates are highly correlated or the objective function is not well approximated by a quadratic.

Now that automatic analytic derivatives are available, DUD is not a recommended method but is retained for backward compatibility.

If you specify SMETHOD=GOLDEN, the step size
*k* is determined by a golden section search.
The parameter TAU determines the length of the initial
interval to be searched, with the interval having length TAU
or 2 ×TAU, depending on SSE.The RHO parameter specifies how fine the search is to be.
The SSE at each endpoint of the interval is
evaluated, and a new subinterval is chosen.
The size of the interval is reduced
until its length is less than RHO.
One pass through the data is required
each time the interval is reduced.
Hence, if RHO is very small relative to TAU, a large
amount of time can be spent determining a step size.
For more information on the GOLDEN search,
refer to Kennedy and Gentle (1980).

If you specify SMETHOD=CUBIC, the NLIN procedure performs a cubic interpolation to estimate the step size. If the estimated step size does not result in a decrease in SSE, step halving is used.

Chapter Contents |
Previous |
Next |
Top |

Copyright © 1999 by SAS Institute Inc., Cary, NC, USA. All rights reserved.