This is documentation for Mathematica 6, which was
based on an earlier version of the Wolfram Language.
 Mathematica Tutorial

# Nonlinear Conjugate Gradient Methods

The basis for a nonlinear conjugate gradient method is to effectively apply the linear conjugate gradient method, where the residual is replaced by the gradient. A model quadratic function is never explicitly formed, so it is always combined with a "line search" method.
The first nonlinear conjugate gradient method was proposed by Fletcher and Reeves as follows. Given a step direction pk, use the line search to find k such that xk+1 = xk + kpk. Then compute
It is essential that the line search for choosing k satisfies the strong Wolfe conditions; this is necessary to ensure that the directions pk are descent directions [NW99]].
An alternate method, which generally (but not always) works better in practice, is that of Polak and Ribiere, where equation (2) is replaced with
In formula (3), it is possible that k+1 can become negative, in which case Mathematica uses the algorithm modified by using pk+1=max (k+1, 0) pk-f (xk+1). In Mathematica, the default conjugate gradient method is Polak-Ribiere, but the Fletcher-Reeves method can be chosen by using the method option