Nonlinear Conjugate Gradient Methods (Mathematica Tutorial)

The basis for a nonlinear conjugate gradient method is to effectively apply the linear conjugate gradient method, where the residual is replaced by the gradient. A model ...

Gauss–Newton Methods (Mathematica Tutorial)

For minimization problems for which the objective function is a sum of squares, it is often advantageous to use the special structure of the problem. Time and effort can be ...

Introduction to Local Minimization (Mathematica Tutorial)

The essence of most methods is in the local quadratic model that is used to determine the next step. The FindMinimum function in Mathematica has five essentially different ...

Introduction to Solving Nonlinear ... (Mathematica Tutorial)

There are some close connections between finding a "local minimum" and solving a set of nonlinear equations. Given a set of n equations in n unknowns, seeking a solution r(x) ...

Introduction to Step Control (Mathematica Tutorial)

Even with "Newton methods" where the local model is based on the actual Hessian, unless you are close to a root or minimum, the model step may not bring you any closer to the ...

Line Search Methods (Mathematica Tutorial)

A method like "Newton's" method chooses a step, but the validity of that step only goes as far as the Newton quadratic model for the function really reflects the function. ...

Newton's Method (Mathematica Tutorial)

Newton's method for nonlinear equations is based on a linear approximation so the Newton step is found simply by setting M_k(p)=0, Near a root of the equations, Newton's ...

Plotting Search Data (Mathematica Tutorial)

The utility functions FindMinimumPlot and FindRootPlot show search data for FindMinimum and FindRoot for one- and two-dimensional functions. They work with essentially the ...

Principal Axis Method (Mathematica Tutorial)

"Gauss–Newton" and "conjugate gradient" methods use derivatives. When Mathematica cannot compute symbolic derivatives, finite differences will be used. Computing derivatives ...

Quasi-Newton Methods (Mathematica Tutorial)

There are many variants of quasi-Newton methods. In all of them, the idea is to base the matrix B_k in the quadratic model on an approximation of the Hessian matrix built up ...