Gauss–Newton Methods (Mathematica Tutorial)

For minimization problems for which the objective function is a sum of squares, it is often advantageous to use the special structure of the problem. Time and effort can be ...

Introduction to Unconstrained ... (Mathematica Tutorial)

Mathematica has a collection of commands that do unconstrained optimization (FindMinimum and FindMaximum) and solve nonlinear equations (FindRoot) and nonlinear fitting ...

Introduction to Step Control (Mathematica Tutorial)

Even with "Newton methods" where the local model is based on the actual Hessian, unless you are close to a root or minimum, the model step may not bring you any closer to the ...

Line Search Methods (Mathematica Tutorial)

A method like "Newton's" method chooses a step, but the validity of that step only goes as far as the Newton quadratic model for the function really reflects the function. ...

Newton's Method (Mathematica Tutorial)

One significant advantage Mathematica provides is that it can symbolically compute derivatives. This means that when you specify Method->"Newton" and the function is ...

Newton's Method (Mathematica Tutorial)

Newton's method for nonlinear equations is based on a linear approximation so the Newton step is found simply by setting M_k(p)=0, Near a root of the equations, Newton's ...

Plotting Search Data (Mathematica Tutorial)

The utility functions FindMinimumPlot and FindRootPlot show search data for FindMinimum and FindRoot for one- and two-dimensional functions. They work with essentially the ...

Principal Axis Method (Mathematica Tutorial)

"Gauss–Newton" and "conjugate gradient" methods use derivatives. When Mathematica cannot compute symbolic derivatives, finite differences will be used. Computing derivatives ...

Quasi-Newton Methods (Mathematica Tutorial)

There are many variants of quasi-Newton methods. In all of them, the idea is to base the matrix B_k in the quadratic model on an approximation of the Hessian matrix built up ...

The Secant Method (Mathematica Tutorial)

When derivatives cannot be computed symbolically, "Newton's" method will be used, but with a finite difference approximation to the Jacobian. This can have costs in terms of ...