**Statistics****`****NonlinearFit****`**

The built-in function Fit finds a least-squares fit of data to a model that is a linear combination of given basis functions; the coefficients of the linear combination are the parameters of the fit. Often a more sophisticated model is desired, where the model is not linear in the parameters. The function NonlinearFit allows one to perform this kind of least-squares fit. NonlinearFit produces an expression for the model, similar to the output of the built-in function Fit. The function NonlinearRegress produces a number of regression diagnostics and allows one to specify exactly what will be included in the output, as does the package function Regress.

The NonlinearFit and NonlinearRegress functions.

The model argument of NonlinearFit and NonlinearRegress must be completely specified by the symbols given as variables and the symbols given as parameters. The symbols given as variables correspond to the independent variables represented in the data.

The data argument can be a list of vectors, each vector consisting of the observed values of the independent variables and the associated response. The data can also be a vector of data points, in which case NonlinearFit or NonlinearRegress assumes that the vector represents the values of a response variable, with the independent variable having values ,

, ... .

Ways of expressing data for NonlinearFit and NonlinearRegress.

The parameters of the nonlinear model are chosen to minimize the merit function given by the sum of squared residuals . The optimization methods used by NonlinearFit and NonlinearRegress are iterative so starting points are required for the parameter search. The most simple way of specifying a parameter is as a symbol, assuming a starting point of 1.0, or as a

symbol,start pair. A parameter can be specified to lie within a prescribed range using

symbol,start,min,max or

symbol,min,max. When parameters are specified with ranges, but no starting points, the starting points are taken to be those values minimizing the merit function out of the set forming a factorial design based on the parameter ranges, where is the number of parameters. For example, if the parameter list is specified as

a,0,1

,

b,0,3

, then the value of

a,b in

1/3,1

,

1/3,2

,

2/3,1

,

2/3,2

that yields the minimum gives starting points for parameters and . Note that the optimization methods used by NonlinearFit and NonlinearRegress locate local minima in the

merit function, so careful choice of starting points may be necessary.

When the parameter range is given, the search for the parameter terminates if the search goes outside the desired range.

This loads the package.
In[1]:= **<< Statistics`NonlinearFit`**

This data set [Meyer and Roth, 1972] gives five values for , , and , describing a reaction involving the catalytic dehydration of n-hexyl alcohol. Here is partial pressure of alchohol, is olefin, and

is the rate of reaction.
In[2]:= **data = {{1.0, 1.0, .126}, {2.0, 1.0, .219},**

{1.0, 2.0, .076}, {2.0, 2.0, .126}, {.1, .0, .186}};

This finds the values of the parameters , , and , such that the model best fits the data triples in a least-squares sense. The least-squares parameters are substituted into the model, just as in Fit

.
In[3]:= **NonlinearFit[data,**

theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2),

{x1, x2}, {theta1, theta2, theta3}]

Out[3]=

NonlinearRegress can be used to produce a list of replacement rules for the parameters.
In[4]:= **BestFitParameters /. NonlinearRegress[data,**

theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2),

{x1, x2}, {theta1, theta2, theta3},

RegressionReport -> BestFitParameters]

Out[4]=

Options for NonlinearFit and NonlinearRegress.

A number of options are common to both NonlinearFit and NonlinearRegress. The option MaxIterations places a limit on the number of steps the algorithm can take. The option ShowProgress controls the printing of the status of the search at each iteration as the search progresses. The Method option allows one to choose between algorithms for performing the minimization of the merit function. The LevenbergMarquardt method gradually shifts the search for the minimum of from steepest descent to quadratic minimization (Gauss-Newton). The FindMinimum method uses the built-in function FindMinimum to minimize , so it corresponds to pure steepest descent.

When using FindMinimum, the gradient functions may be specified using the Gradient option. It is also possible to specify a parameter as

symbol,

,

or

symbol,

,

,min,max, in which case FindMinimum will calculate derivatives with respect to the parameters numerically.

The option Weights allows one to specify a list of weights, one for each data point; the default is a weight of unity for each data point. Weights can also specify a pure function of one argument; the function is applied to the response value to determine the weight. Thus specifying Weights

->

, ... ,

means that the parameters are chosen to minimize the weighted sum of residuals . Specifying Weights

->Sqrt[#]& means that the parameters are chosen to minimize

.

Options unique to NonlinearRegress.

The option RegressionReport is accepted by NonlinearRegress, but not NonlinearFit. If RegressionReport is not specified, NonlinearRegress automatically produces a list including values for BestFitParameters, ParameterCITable, EstimatedVariance, ANOVATable, AsymptoticCorrelationMatrix, and FitCurvatureTable. This set of statistics comprises the default SummaryReport.

NonlinearRegress will also accept any option that can be specified for StudentTCI. In particular, ConfidenceLevel specifies the reference parameter confidence region (the relative curvature of which is given in FitCurvatureTable) and the individual parameter confidence intervals given in ParameterCITable.

Here the progress trace is enabled.
In[5]:= **NonlinearRegress[data,**

theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2),

{x1, x2}, {theta1, theta2, theta3},

RegressionReport -> BestFitParameters,

ShowProgress -> True]

Iteration:1 ChiSquared:0.236342 Parameters:{1., 1., 1.}

Iteration:2 ChiSquared:0.0290303 Parameters:{2.52778, 3.24966, 0.240862}

Iteration:3 ChiSquared:0.00376852 Parameters:{8.21167, 16.0978, 0.482639}

Iteration:4 ChiSquared:0.000169052 Parameters:{5.32375, 15.497, 0.540802}

Iteration:5 ChiSquared:0.000111339 Parameters:{4.72603, 15.3973, 0.579105}

Iteration:6 ChiSquared:0.000100974 Parameters:{4.02948, 15.2982, 0.637589}

Iteration:7 ChiSquared:0.000058774 Parameters:{3.82689, 15.2993, 0.671561}

Iteration:8 ChiSquared:0.000052346 Parameters:{3.55752, 15.2434, 0.706181}

Iteration:9 ChiSquared:0.0000471917 Parameters:{3.47193, 15.2246, 0.722035}

Iteration:10 ChiSquared:0.0000456348 Parameters:{3.22399, 15.1761, 0.760263}

Iteration:11 ChiSquared:0.0000436003 Parameters:{3.13708, 15.1601, 0.778527}

Iteration:12 ChiSquared:0.0000435527 Parameters:{3.13169, 15.1594, 0.780025}

Iteration:13 ChiSquared:0.0000435527 Parameters:{3.13151, 15.1594, 0.780062}

Out[5]=

By not specifying RegressionReport, the default regression output is produced.
In[6]:= **NonlinearRegress[data,**

theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2),

{x1, x2}, {theta1, theta2, theta3}]

Out[6]=

The option RegressionReport can be used to specify a single statistic or a list of statistics so that more (or less) than the default set of statistics is included in the output. RegressionReportValues[NonlinearRegress] gives the values that may be included in the RegressionReport list for the NonlinearRegress function.

Possible settings for RegressionReport or settings that may be included in a list specified by RegressionReport.

ANOVATable provides a partitioning of the sum of squares. Both the uncorrected total sum of squares (without intercept term) and the corrected total sum of squares (with intercept term) are included. The corrected total sum of squares is useful for calculating a value for comparable to that produced by Regress, so that the fit of a nonlinear model can be compared to the fit of a linear model. For example, one possible definition of for a nonlinear regression is the ratio of the difference between the corrected sum of squares and the model sum of squares and the corrected sum of squares. The remaining statistics are similar to their counterparts produced by the linear regression function Regress

.

Objects describing the search for the least-squares fit.

Including StartingParameters in the list of statistics requested by RegressionReport is helpful if the search for a least-squares fit is unsuccessful and it is necessary to restart the search at a new point.

Diagnostics for detecting outliers.

If the nonlinear model is approximately linear at the least-squares fit, HatDiagonal and StandardizedResiduals are useful for detecting outliers in the data. HatDiagonal gives the leverage of each point, measuring whether each observation of the predictor variables is unusual; a leverage of zero indicates no influence, while a leverage of one indicates that a degree of freedom has been lost to fitting that point.

Diagnostics for evaluating the validity of a linear model approximation to the nonlinear model.

FitCurvatureTable breaks the relative curvature of the solution locus at the least-squares estimate into two components, intrinsic and parameter-effects. For the combination of data and model given above, and . The solution locus is a -dimensional subspace of given by

, . Intrinsic curvature describes the normal component of the solution locus curvature at the least-squares estimate. Parameter-effects curvature describes the tangential component of the solution locus curvature at the least-squares estimate. Standardizing curvature to be response-invariant gives relative curvature. Maximizing the relative intrinsic curvature over the -dimensional subspace normal to the locus gives the maximum relative intrinsic curvature. Similarly, maximizing the relative parameter-effects curvature over the

-dimensional subspace tangential to the locus gives the maximum relative parameter-effects curvature.

Both of these quantities can be compared to the relative curvature of the confidence region centered on the least-squares parameter estimates. If the maximum relative intrinsic curvature is small compared to the confidence region relative curvature, the solution locus is approximately linear over the confidence region. If the maximum relative parameter-effects curvature is small compared to the confidence region relative curvature, the parameter coordinates projected onto the tangential plane are approximately parallel and uniformly spaced over the confidence region.

ParameterBias

is based on the average curvature of the solution locus tangential to the least-squares estimate.

Here the same data is fit to a reparametrized model, Only the BestFit and the FitCurvatureTable

are output.
In[7]:= **NonlinearRegress[data,**

x1 / (phi0 + phi1 x1 + phi2 x2),

{x1, x2}, {phi0, phi1, phi2},

RegressionReport -> {BestFit, FitCurvatureTable}]

Out[7]=

Note that reparametrizing the model does not affect the intrinsic curvature. However, in this case, reparametrization has reduced the parameter-effects curvature below the critical value of 0.22843. This indicates that the least-squares estimates of the parameters , , and have "nearly linear" behavior. This could not be said about the least-squares estimates of the parameters , , and in the original model [Ratkowsky, 1983]. The maximum relative curvature diagnostics are useful if you wish to make inferences based on the linear approximation to the nonlinear model. For example, diagnostics based on the asymptotic standard errors of the parameters are of questionable validity when the linear approximation to the model is poor at the least-squares parameter estimates and the sample size

is small.

**References**

Meyer, R. R. and Roth, P. M. (1972). "Modified Damped Least Squares: An Algorithm for Nonlinear Estimation," J. Inst. Math. Appl. 9, 218-233.

Ratkowsky, David A. (1983). Nonlinear Regression Modeling, A Unified Practical Approach, Marcel Dekker, New York.