Chapter Contents |
Previous |
Next |

The NLP Procedure |

In each iteration *k*, the (dual) quasi-Newton, hybrid quasi-Newton,
conjugate gradient, and Newton-Raphson minimization techniques use
iterative line-search algorithms that try to optimize a linear,
quadratic, or cubic approximation of *f* along a feasible descent
search direction *s*^{(k)}

Therefore, a line-search algorithm is an iterative process
that optimizes a nonlinear function of one parameter
() within each iteration *k* of the optimization technique,
which itself tries to optimize a linear or quadratic approximation
of the nonlinear objective function *f*=*f*(*x*) of *n* parameters *x*.
Since the outside iteration process is based only on the
approximation of the objective function, the inside iteration
of the line-search algorithm does not have to be perfect.
Usually, the choice of significantly
reduces (in a minimization) the objective function. Criteria often
used for termination of line-search algorithms are the Goldstein conditions
(refer to Fletcher 1987).

Various line-search algorithms can be selected by using the LIS= option. The line-search method LIS=2 seems to be superior when function evaluation consumes significantly less computation time than gradient evaluation. Therefore, LIS=2 is the default value for Newton-Raphson, (dual) quasi-Newton, and conjugate gradient optimizations.

A special default line-search algorithm for TECH= HYQUAN is
useful only for least-squares problems and cannot be chosen
by the LIS= option. This method uses three columns of the
*m* ×*n* Jacobian matrix, which can for large *m* require
more memory than using the algorithms designated by LIS=1
through LIS=8.

The line-search methods LIS=2 and LIS=3 can be modified to exact line-search by using the LSPRECISION= option (specifying the parameter in Fletcher, 1987). The line-search methods LIS=1, LIS=2, and LIS=3 satisfy the left-hand side and right-hand side Goldstein conditions (refer to Fletcher 1987). When derivatives are available, the line-search methods LIS=6, LIS=7,and LIS=8 try to satisfy the right-hand side Goldstein condition; if derivatives are not available, these line-search algorithms use only function calls.

Chapter Contents |
Previous |
Next |
Top |

Copyright © 1999 by SAS Institute Inc., Cary, NC, USA. All rights reserved.