Statistics`NonlinearFit`
The builtin function Fit finds a leastsquares fit to a list of data for a model that is a linear combination of the given basis functions; the coefficients of the linear combination are the parameters of the fit. Often a more sophisticated model is desired, where the model is not linear in the parameters. The function NonlinearFit allows you to perform this kind of leastsquares fit. NonlinearFit gives an expression for the model, similar to the output of the builtin function Fit. The function NonlinearRegress gives a number of regression diagnostics and allows you to specify exactly what will be included in the output, as does the package function Regress.
The NonlinearFit and NonlinearRegress functions.
The model argument of NonlinearFit and NonlinearRegress must be completely specified by the symbols in the variables argument and the symbols in the parameters argument. The variables argument specifies the independent variables represented in the data. The parameters argument specifies the model parameters for which you would like estimates.
The data argument can be a list of vectors, each vector consisting of the values of the independent variables , , ... , followed by the observed value of the associated response . The data argument can also be a vector, in which case it is assumed that the vector represents the observed values of the response variable, with the independent variable(s) for the response equal to .
Ways of specifying data in NonlinearFit and NonlinearRegress.
The estimates of the model parameters are chosen to minimize the merit function given by the sum of squared residuals . The optimization methods used by NonlinearFit and NonlinearRegress are iterative so starting values are required for the parameter estimate search. Careful choice of starting values may be necessary, as the parameter estimates found by NonlinearFit and NonlinearRegress may represent a local minimum in the merit function.
The simplest way to specify a parameter is as a symbol, assuming a starting point of 1.0, or as a symbol, start pair. When elements in the parameter are specified as symbol, min, max, the starting parameter values are taken to be those minimizing the merit function out of the set forming a factorial design based on the parameter ranges, where is the number of parameters. For example, if the parameter list is specified as a, 0, 1, b, 0, 3, then the value of a, b in 1/3, 1, 1/3, 2, 2/3, 1, 2/3, 2 that yields the minimum gives starting values for parameters and .
If a parameter is specified as symbol, , , then the search for parameter estimates uses and as the first two values of symbol. This form must be used if symbolic derivatives of with respect to the parameters cannot be found.
This loads the package.
In[1]:= << Statistics`NonlinearFit`
This data set [Meyer and Roth, 1972] gives five values for , , and , describing a reaction involving the catalytic dehydration of nhexyl alcohol. Here is partial pressure of alchohol, is olefin, and is the rate of reaction.
In[2]:= data = {{1.0, 1.0, .126}, {2.0, 1.0, .219}, {1.0, 2.0, .076}, {2.0, 2.0, .126}, {.1, .0, .186}};
This finds the values of the parameters , , and , such that the model best fits the data triples in a leastsquares sense. The leastsquares parameters are substituted into the model, just as in Fit.
In[3]:= NonlinearFit[data, theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2), {x1, x2}, {theta1, theta2, theta3}]
Out[3]=
NonlinearRegress can be used to produce a list of replacement rules for the parameters.
In[4]:= BestFitParameters /. NonlinearRegress[data, theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2), {x1, x2}, {theta1, theta2, theta3}, RegressionReport > BestFitParameters]
Out[4]=
Options for NonlinearFit and NonlinearRegress.
NonlinearFit and NonlinearRegress use FindMinimum to find parameter estimates, thus both functions accept the FindMinimum options AccuracyGoal, Compiled, Gradient, MaxIterations, Method, PrecisionGoal, and WorkingPrecision. The Method option allows you to choose between algorithms for performing the minimization of the merit function. The LevenbergMarquardt method gradually shifts the search for the minimum of from steepest descent to quadratic minimization. Other possible settings for Method are Gradient (steepest descent), Newton, QuasiNewton, and Automatic. The Automatic method does linear fitting for linear models and LevenbergMarquardt search for nonlinear models, thus the meaning of the automatic method differs between NonlinearFit (or NonlinearRegress) and FindMinimum.
The Weights option allows you to implement weighted least squares by specifying a list of weights, one for each data point; the default Weights > Automatic implies a weight of unity for each data point. When Weights > , ... , , the parameter estimates are chosen to minimize the weighted sum of squared residuals .
Weights can also specify a pure function of the response. For example, to choose parameter estimates to minimize , set Weights > (Sqrt[#] &).
Options unique to NonlinearRegress.
The option RegressionReport is accepted by NonlinearRegress, but not NonlinearFit. If RegressionReport is not specified, NonlinearRegress automatically gives a list including values for BestFitParameters, ParameterCITable, EstimatedVariance, ANOVATable, AsymptoticCorrelationMatrix, and FitCurvatureTable. This set of objects comprises the default SummaryReport. The option RegressionReport can be used to specify a single object or a list of objects so that more (or less) than the default set of results is included in the output. RegressionReportValues[NonlinearRegress] gives the objects that may be included in the RegressionReport list for the NonlinearRegress function.
NonlinearRegress will also accept any option that can be specified for StudentTCI. In particular, ConfidenceLevel specifies the reference parameter confidence region (the relative curvature of which is given in FitCurvatureTable) and the individual parameter confidence intervals given in ParameterCITable.
If the RegressionReport option is not specified, the default regression output is produced. Note that the AsymptoticCorrelationMatrix indicates that the parameters and are highly correlated, a negative aspect of this model.
In[5]:= NonlinearRegress[data, theta1 theta3 x1 / (1 + theta1 x1 + theta2 x2), {x1, x2}, {theta1, theta2, theta3}]
Out[5]=
Some option settings for RegressionReport or objects that may be included in a list specified by RegressionReport.
ANOVATable provides a partitioning of the sum of squares. Both the uncorrected total sum of squares (without intercept term) and the corrected total sum of squares (with intercept term) are included. The corrected total sum of squares is useful for calculating a value for comparable to that produced by Regress, so that the fit of a nonlinear model can be compared to the fit of a linear model. For example, one possible definition of for a nonlinear regression is the ratio of the difference between the corrected total sum of squares and the residual sum of squares to the corrected total sum of squares. The other objects in this table have meanings similar to what they would have if specified by the RegressionReport option of the linear regression function Regress.
Objects describing the search for the leastsquares fit.
Including StartingParameters in the list of objects requested by RegressionReport is helpful if the search for a leastsquares fit is unsuccessful and it is necessary to restart the search at a new point.
Diagnostics for detecting outliers.
If the nonlinear model is approximately linear at the leastsquares fit, HatDiagonal and StandardizedResiduals are useful for detecting outliers in the data. HatDiagonal gives the leverage of each point, measuring whether each observation of the predictor variables is unusual; a leverage of zero indicates no influence, while a leverage of one indicates that a degree of freedom has been lost to fitting that point.
Diagnostics for evaluating the validity of a linear model approximation to the nonlinear model.
FitCurvatureTable expresses the relative curvature of the solution locus at the leastsquares estimate in terms of two components, intrinsic and parametereffects. For the combination of data and model given above, and . The solution locus is a dimensional subspace of given by , .
Intrinsic curvature describes the normal component of the solution locus curvature at the leastsquares estimate. Parametereffects curvature describes the tangential component of the solution locus curvature at the leastsquares estimate. Standardizing curvature to be responseinvariant gives relative curvature. Maximizing the relative intrinsic curvature over the dimensional subspace normal to the locus gives the maximum relative intrinsic curvature. Similarly, maximizing the relative parametereffects curvature over the dimensional subspace tangential to the locus gives the maximum relative parametereffects curvature.
Both of these quantities can be compared to the relative curvature of the confidence region centered on the leastsquares parameter estimates. If the maximum relative intrinsic curvature is small compared to the confidence region relative curvature, the solution locus is approximately planar over the confidence region. If the maximum relative parametereffects curvature is small compared to the confidence region relative curvature, the parameter coordinates projected onto the tangential plane are approximately parallel and uniformly spaced over the confidence region.
ParameterBias is based on the average curvature of the solution locus tangential to the leastsquares estimate.
Here the same data is fit to a reparametrized model, Only the BestFit, AsymptoticCorrelationMatrix, and FitCurvatureTable are calculated.
In[6]:= NonlinearRegress[data, x1 / (phi0 + phi1 x1 + phi2 x2), {x1, x2}, {phi0, phi1, phi2}, RegressionReport > {BestFit, AsymptoticCorrelationMatrix, FitCurvatureTable}]
Out[6]=
Note that reparametrizing the model does not affect the intrinsic curvature. However, here reparametrization has reduced the parametereffects curvature below the critical value of 0.22843. This indicates that the leastsquares estimates of the parameters , , and have nearly linear behavior. This could not be said about the leastsquares estimates of the parameters , , and in the original model [Ratkowsky, 1983]. Unfortunately, now the asymptotic correlation between parameters is high for all pairs; a model with fewer parameters should be considered.
The maximum relative curvature diagnostics are useful if you wish to make inferences based on the linear approximation to the nonlinear model. For example, diagnostics based on the asymptotic standard errors of the parameters, such as ParameterCITable and ParameterConfidenceRegion, are of questionable validity when the linear approximation to the model is poor at the leastsquares parameter estimates. The linear approximation is based on the planar assumption, tested by maximum relative intrinsic curvature, and the uniform coordinate assumption, tested by maximum relative parametereffects curvature. It is usually the uniform coordinate assumption that is invalid [Bates and Watts, 1988].
References
Bates, D. M. and Watts, D. G. (1988). Nonlinear Regression Analysis and Its Applications, John Wiley & Sons, New York.
Meyer, R. R. and Roth, P. M. (1972). "Modified Damped Least Squares: An Algorithm for Nonlinear Estimation," J. Inst. Math. Appl. 9, 218233.
Ratkowsky, David A. (1983). Nonlinear Regression Modeling, A Unified Practical Approach, Marcel Dekker, New York.
