In simple linear regression, the F-statistic is used to test the null hypothesis $$H_0$$ that there is no relationship between the predictor and the response. If the null hypothesis is true, the F-statistic follows an F-distribution with 1 and n-2 degrees of freedom: $$ F = \frac{(TSS - RSS) / 1}{RSS / (n-2)} $$ where TSS is the total sum of squares and RSS is the residual sum of squares.

University of Michigan - Ann Arbor

Google

The extent to which the linear regression model fits the data, the quality of linear regression fit, can be achieved by:
1. $ R^2 $
2. Residual Standard Error
3. Goodnesss of Fit
4. F Statistics
5. Mean Square Error (MSE)

Assessing the accuracy of linear regression

Residual Standard Error is an estimate of σ standard deviation. It assesses the quality of linear regression. It is the average deviation of a response from the true regression line.

RSE is considered a measure of lack of fit. So if it is large, the model is considered not to fit well whereas if RSE is small, the model may fit well.

$$ RSE = \sqrt{\dfrac{RSS}{n-2}}= \sqrt{\dfrac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{n-2}} $$
Where n − 2 is the degrees of freedom.

Residual Standard Error

Goodness of fit is a measure of the strength of the linear relationship between two variables. The regression line $$\hat{Y}_t = \hat{\alpha} + \hat{\beta} X_t$$ obtained by the least squares method is the best fitting line of all possible lines at least in the sense that the sum of squares of residuals is the smallest.

Goodness of Fit

Mean Square Error (MSE) reflects the degree of difference between the estimated value and the actual value. If the predicted response value is close to the real response value, MSE will be very small. If the predicted response value is substantially different from the real response value, MSE will be very large.
$$ MSE = \frac{1}{m} \sum_{i=1}^{m} (y_m - \hat{y_m})^2 $$


Mean Square Error (MSE) - Evaluation Model

R-Squared ($R^2$)—the proportion of variance explained—measures how well a prediction model for "regression" fits the given data. It is independent of the scale of $Y$.
$$0 \leq R^2 \leq 1$$
The larger the R-Squared value, the closer it is to 1, the better the regression model explains the variation of observations around its mean.
$$ R^2 = \frac{TSS-RSS}{TSS} = 1 - \frac{RSS}{TSS} = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2} $$

Learn Before

Related