LinearModelFit

LinearModelFit[{{x1,y1},{x2,y2},},{f1,f2,},x]

constructs a linear model of the form that fits the yi for successive xi values.

LinearModelFit[data,{f1,f2,},{x1,x2,}]

constructs a linear model where the fi depend on the variables xk.

LinearModelFit[{m,v}]

constructs a linear model from the design matrix m and response vector v.

Details and Options

  • LinearModelFit attempts to model the input data using a linear combination of functions.
  • LinearModelFit produces a linear model of the form under the assumption that the original are independent normally distributed with mean and common standard deviation.
  • LinearModelFit returns a symbolic FittedModel object to represent the linear model it constructs.
  • The value of the best-fit function from LinearModelFit at a particular point x1, can be found from model[x1,].
  • Possible forms of data are:
  • {y1,y2,}equivalent to the form {{1,y1},{2,y2},}
    {{x11,x12,,y1},}a list of independent values xij and the responses yi
    {{x11,x12,}y1,}a list of rules between input values and response
    {{x11,x12,},}{y1,y2,}a rule between a list of input values and of responses
    {{x11,,y1,},}nfit the n^(th) column of a matrix
  • With multivariate data such as {{x_(11),x_(12),... ,y_(1)},{x_(21),x_(22),... ,y_(2)},...}, the number of coordinates xi1, xi2, should equal the number of variables xi.
  • Additionally, data can be specified using a design matrix without specifying functions and variables:
  • {m,v}a design matrix m and response vector v
  • In LinearModelFit[{m,v }], the design matrix m is formed from the values of basis functions fi at data points in the form {{f1,f2,},{f1,f2,},}. The response vector v is the list of responses {y1,y2,}.
  • When a design matrix is used, the basis functions fi can be specified using the form LinearModelFit[{m,v},{f1,f2,}].
  • For a design matrix m and response vector v, the model is , where is the vector of parameters to be estimated.
  • Options
  • LinearModelFit takes the following options:
  • ConfidenceLevel 95/100confidence level to use for parameters and predictions
    IncludeConstantBasis Truewhether to include a constant basis function
    LinearOffsetFunction Noneknown offset in the linear predictor
    NominalVariables Nonevariables considered as nominal or categorical
    VarianceEstimatorFunction Automaticfunction for estimating the error variance
    Weights Automaticweights for data elements
    WorkingPrecision Automaticprecision used in internal computations
  • With the setting IncludeConstantBasis->False, a model of the form is fitted. The option IncludeConstantBasis is ignored if the design matrix is specified in the input.
  • With the setting LinearOffsetFunction->h, a model of the form is fitted.
  • With ConfidenceLevel->p, probability-p confidence intervals are computed for parameter and prediction intervals.
  • With the setting Weights->Automatic, the weights will be set to 1 if the responses are exact values. If the responses are all of the form Around[yi,σi], the weights will be set to .
  • With the setting Weights->{w1,w2,}, the error variance for yi is assumed to be . By default, unit weights are used.
  • With the setting VarianceEstimatorFunction->f, the variance is estimated by f[res,w], where res={y1-,y2-,} is the list of residuals and w={w1,w2,} is the list of weights for the measurements yi.
  • Using VarianceEstimatorFunction->(1&) and Weights->{1/Δy12,1/Δy22,}, Δyi is treated as the known uncertainty of measurement yi and parameter standard errors are effectively computed only from the weights.
  • Properties
  • The properties and diagnostics of the FittedModel can be obtained from model["property"].
  • Properties related to data and the fitted function obtained using model["property"] include:
  • "BasisFunctions"list of basis functions
    "BestFit"fitted function
    "BestFitParameters"parameter estimates
    "Data"the input data or design matrix and response vector
    "DesignMatrix"design matrix for the model
    "Function"best fit pure function
    "Response"response values in the input data
  • Types of residuals include:
  • "FitResiduals"difference between actual and predicted responses
    "StandardizedResiduals"fit residuals divided by the standard error for each residual
    "StudentizedResiduals"fit residuals divided by single deletion error estimates
  • Properties related to the sum of squared errors include:
  • "ANOVATable"analysis of variance table
    "ANOVATableDegreesOfFreedom"degrees of freedom from the ANOVA table
    "ANOVATableEntries"unformatted array of values from the table
    "ANOVATableFStatistics"F-statistics from the table
    "ANOVATableMeanSquares"mean square errors from the table
    "ANOVATablePValues"values from the table
    "ANOVATableSumsOfSquares"sums of squares from the table
    "CoefficientOfVariation"estimated standard deviation divided by the response mean
    "EstimatedVariance"estimate of the error variance
    "PartialSumOfSquares"changes in model sum of squares as nonconstant basis functions are removed
    "SequentialSumOfSquares"the model sum of squares partitioned componentwise
  • Properties and diagnostics for parameter estimates include:
  • "CorrelationMatrix"parameter correlation matrix
    "CovarianceMatrix"parameter covariance matrix
    "EigenstructureTable"eigenstructure of the parameter correlation matrix
    "EigenstructureTableEigenvalues"eigenvalues from the table
    "EigenstructureTableEntries"unformatted array of values from the table
    "EigenstructureTableIndexes"index values from the table
    "EigenstructureTablePartitions"partitioning from the table
    "ParameterTable"table of fitted parameter information
    "ParameterConfidenceIntervals"parameter confidence intervals
    "ParameterConfidenceIntervalTable"table of confidence interval information for the fitted parameters
    "ParameterConfidenceIntervalTableEntries"unformatted array of values from the table
    "ParameterConfidenceRegion"ellipsoidal parameter confidence region
    "ParameterErrors"standard errors for parameter estimates
    "ParameterPValues"values for parameter statistics
    "ParameterTableEntries"unformatted array of values from the table
    "ParameterTStatistics"statistics for parameter estimates
    "VarianceInflationFactors"list of inflation factors for the estimated parameters
  • Properties related to influence measures include:
  • "BetaDifferences"DFBETAS measures of influence on parameter values
    "CatcherMatrix"catcher matrix
    "CookDistances"list of Cook distances
    "CovarianceRatios"COVRATIO measures of observation influence
    "DurbinWatsonD"DurbinWatson statistic for autocorrelation
    "FitDifferences"DFFITS measures of influence on predicted values
    "FVarianceRatios"FVARATIO measures of observation influence
    "HatDiagonal"diagonal elements of the hat matrix
    "SingleDeletionVariances"list of variance estimates with the ^(th) data point omitted
  • Properties of predicted values include:
  • "MeanPredictionBands"confidence bands for mean predictions
    "MeanPredictionConfidenceIntervals"confidence intervals for the mean predictions
    "MeanPredictionConfidenceIntervalTable"table of confidence intervals for the mean predictions
    "MeanPredictionConfidenceIntervalTableEntries"unformatted array of values from the table
    "MeanPredictionErrors"standard errors for mean predictions
    "PredictedResponse"fitted values for the data
    "SinglePredictionBands"confidence bands based on single observations
    "SinglePredictionConfidenceIntervals"confidence intervals for the predicted response of single observations
    "SinglePredictionConfidenceIntervalTable"table of confidence intervals for the predicted response of single observations
    "SinglePredictionConfidenceIntervalTableEntries"unformatted array of values from the table
    "SinglePredictionErrors"standard errors for the predicted response of single observations
  • Properties that measure goodness of fit include:
  • "AdjustedRSquared" adjusted for the number of model parameters
    "AIC"Akaike Information Criterion
    "AICc"finite sample corrected AIC
    "BIC"Bayesian Information Criterion
    "RSquared"coefficient of determination
  • For the properties "RSquared" and "AdjustedRSquared", the computation of the total sum of squares is mean adjusted only when the constant basis is included.

Examples

open allclose all

Basic Examples  (1)

Fit a linear model to some data:

Obtain the functional form:

Evaluate the model at a point:

Visualize the fitted function with the data:

Extract information about the fitting:

Plot the residuals:

Scope  (17)

Data  (7)

Fit a model of one variable assuming increasing integer independent values:

This is equivalent to:

Fit a model of more than one variable, assuming the response is the last one:

This is equivalent to:

Specify a column as the response:

Fit a list of rules:

Fit a rule of input values and responses:

Fit a model with categorical predictor variables:

Fit a model given a design matrix and response vector:

Fit the model referring to the basis functions as x and y:

Model  (3)

Find the best fit linear coefficient for a function:

Fit data to a linear combination of linear functions of independent variables:

This is equivalent to explicitly specifying a constant function:

Fit data to a linear combination of nonlinear functions of independent variables:

Properties  (7)

Data & Fitted Functions  (1)

Fit a linear model:

Extract the original data:

Obtain and plot the best fit:

Obtain the fitted function as a pure function:

Get the design matrix and response vector for the fitting:

Residuals  (1)

Examine residuals for a fit:

Visualize the raw fit residuals:

Visualize scaled residuals in stem plots:

Plot the absolute differences between the standardized and Studentized residuals:

Sums of Squares  (1)

Fit a linear model to some data:

Extract the estimated error variance and coefficient of variation:

Obtain an analysis of variance table for the model:

Get the F-statistics from the table:

Extract the numeric entries from the table:

Use Grid to add formatting:

Parameter Estimation Diagnostics  (1)

Obtain a formatted table of parameter information:

Obtain an analysis of variance table for the model:

Get the unformatted array of values from the table:

Add formatting using Grid:

Add formatting via TableForm:

Influence Measures  (1)

Fit some data containing extreme values to a linear model:

Use single deletion variances to check the impact on the error variance of removing each point:

Check Cook distances to identify highly influential points:

Use DFFITS values to assess the influence of each point on the fitted values:

Use DFBETAS values to assess the influence of each point on each estimated parameter:

Prediction Values  (1)

Fit a linear model:

Plot the predicted values against the observed values:

Obtain tabular results for mean and single prediction confidence intervals:

Get the single prediction intervals from the table:

Extract 99% mean prediction bands:

Goodness-of-Fit Measures  (1)

Obtain a table of goodness-of-fit measures for a linear model:

Compute goodness-of-fit measures for all possible linear submodels:

Rank the models by :

Rank the models by adjusted , which penalizes for adding terms:

Generalizations & Extensions  (1)

Perform other mathematical operations on the functional form of the model:

Integrate symbolically and numerically:

Find a predictor value that gives a particular value for the model:

Options  (8)

ConfidenceLevel  (1)

The default gives 95% confidence intervals:

Use 99% intervals instead:

Set the level to 90% within FittedModel:

IncludeConstantBasis  (1)

Fit a simple linear regression model:

Fit the linear model with intercept zero:

LinearOffsetFunction  (1)

Fit data to a linear model:

Fit data to a linear model with a known Sqrt[x] term:

NominalVariables  (1)

Fit data treating the first variable as a nominal variable:

Treat both variables as nominal:

VarianceEstimatorFunction  (1)

Use the default unbiased estimate of error variance:

Assume a known error variance:

Estimate the variance by the mean squared error:

Weights  (2)

Fit a model using equal weights:

Give explicit weights for the data points:

Use Around values to give different weights to data points:

WorkingPrecision  (1)

Use WorkingPrecision to get higher precision in parameter estimates:

Obtain the fitted function:

Reduce the precision in property computations after the fitting:

Applications  (6)

Fit the first 100 primes to a linear model:

Visualize the fit:

The systematic trend in the residuals violates the assumption of independent normal errors:

Fit a linear model of multiple variables:

Visually inspect the residuals by data point:

Plot the residuals against each predictor variable:

Plot Cook's distances to diagnose leverage:

Find the positions of distances above a given cutoff value:

Extract the associated data points:

Use - plots to check the assumption of normal errors:

Compare standardized residuals to standard normal values:

Do the comparison with Studentized residuals:

Simulate some data with a continuous and a nominal variable:

Fit an analysis of covariance model to the data:

Obtain an analysis of variance table for the model:

Group the data by treatment:

Visualize the grouped data and associated curves:

Use properties to compute additional results:

Extract the design matrix and residuals:

Compute White's heteroskedasticity-consistent covariance estimate:

Compare with the covariance assuming homoskedasticity:

Compare standard errors based on the two covariance estimates:

Perform a BreuschPagan test:

Fit a model:

Fit the squared errors to a model with the same predictors:

Compute the BreuschPagan test statistic:

Compute the -value:

Properties & Relations  (9)

DesignMatrix constructs the design matrix used by LinearModelFit:

By default, LinearModelFit and GeneralizedLinearModelFit fit equivalent models:

LinearModelFit fits linear models assuming normally distributed errors:

NonlinearModelFit fits nonlinear models assuming normally distributed errors:

Fit and LinearModelFit fit equivalent models:

LinearModelFit allows for extraction of additional information about the fitting:

Fit a linear model to data:

Perform the same fitting using a design matrix and response vector:

Obtain the parameter estimates via LeastSquares:

LinearModelFit fits linear models:

FindFit gives parameter estimates for linear and nonlinear models:

LinearModelFit will use the time stamps of a TimeSeries as variables:

Rescale the time stamps and fit again:

Find fit for the values:

LinearModelFit acts pathwise on a multipath TemporalData:

Do a simple linear model fit:

Do the same fit using a neural net with a single linear layer:

Wolfram Research (2008), LinearModelFit, Wolfram Language function, https://reference.wolfram.com/language/ref/LinearModelFit.html.

Text

Wolfram Research (2008), LinearModelFit, Wolfram Language function, https://reference.wolfram.com/language/ref/LinearModelFit.html.

CMS

Wolfram Language. 2008. "LinearModelFit." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/LinearModelFit.html.

APA

Wolfram Language. (2008). LinearModelFit. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/LinearModelFit.html

BibTeX

@misc{reference.wolfram_2023_linearmodelfit, author="Wolfram Research", title="{LinearModelFit}", year="2008", howpublished="\url{https://reference.wolfram.com/language/ref/LinearModelFit.html}", note=[Accessed: 19-March-2024 ]}

BibLaTeX

@online{reference.wolfram_2023_linearmodelfit, organization={Wolfram Research}, title={LinearModelFit}, year={2008}, url={https://reference.wolfram.com/language/ref/LinearModelFit.html}, note=[Accessed: 19-March-2024 ]}