There are some close connections between finding a
"local minimum" and solving a set of nonlinear equations. Given a set of
n equations in
n unknowns, seeking a solution
is equivalent to minimizing the sum of squares
when the residual is zero at the minimum, so there is a particularly close connection to the
"Gauss-Newton" methods. In fact, the Gauss-Newton step for local minimization and the
"Newton" step for nonlinear equations are exactly the same. Also, for a smooth function,
"Newton's method" for local minimization is the same as Newton's method for the nonlinear equations
. Not surprisingly, many aspects of the algorithms are similar; however, there are also important differences.
Another thing in common with minimization algorithms is the need for some kind of
"step control". Typically, step control is based on the same methods as minimization except that it is applied to a merit function, usually the smooth 2-norm squared,
.
