what are residuals in linear regression

Residuals in linear regression are the “leftover” errors: the difference between what your model predicts and what actually happens in the data.

What are residuals (quick idea)

In a simple linear regression, your model predicts a value y^\hat{y}y^​ for each observed yyy.

The residual for that data point is:

residual e=y−y^\text{residual }e=y-\hat{y}residual e=y−y^​

So:

• If e>0e>0e>0: the actual value is above the regression line.

• If e<0e<0e<0: the actual value is below the regression line.

• If e=0e=0e=0: the point lies exactly on the line.

Each data point in your dataset has exactly one residual.

A quick mental picture

Imagine you draw the best-fit straight line through a cloud of points on a scatter plot.

If you drop a vertical line from each point down (or up) to your regression line, that vertical distance is the residual for that point.

• Short vertical distance → prediction was pretty good.

• Long vertical distance → prediction was far off.

Why residuals matter

Residuals are not just “errors”; they are a diagnostic tool to judge how good your model is.

Key uses:

1. Model accuracy
   • If most residuals are close to 0, your model fits well.

 * Large residuals mean the model is often wrong.

2. Checking assumptions
   In linear regression, we assume errors are roughly:

   • Mean 0 (and in fact, for ordinary least squares, the sum and mean of residuals are 0).

 * Constant variance (no funnel shape when plotting residuals).

 * No strong patterns (a clear curve suggests the relationship isn’t really linear).

A residual plot (residuals vs. fitted values) should look like a random cloud; visible patterns hint that the model is misspecified.

3. Finding outliers and leverage points
   Points with very large residuals may be outliers; they don’t follow the same pattern as most of the data.

4. Model comparison
   In ordinary least squares, the model is chosen to minimize the sum of squared residuals.

Smaller sum of squared residuals usually means a better fit, and this quantity is directly connected to metrics like R2R^2R2.

How you actually compute residuals (tiny example)

Step-by-step:

1. Fit a linear regression and get an equation like
   y^=b0+b1x\hat{y}=b_0+b_1xy^​=b0​+b1​x.

2. For each data point xi,yix_i,y_ixi​,yi​, compute the predicted y^i\hat{y}_iy^​i​ using that equation.

3. Compute ei=yi−y^ie_i=y_i-\hat{y}_iei​=yi​−y^​i​ for each point.

If you sum all these eie_iei​ for an ordinary least squares regression, you get 0 (up to rounding).

Simple HTML table summary

Here is a compact summary in HTML table form, as requested:

html

<table>
  <tr>
    <th>Concept</th>
    <th>Explanation</th>
  </tr>
  <tr>
    <td>Definition</td>
    <td>Residual = observed value (y) minus predicted value (ŷ); a vertical distance from each point to the regression line.[web:1][web:3]</td>
  </tr>
  <tr>
    <td>Sign of residual</td>
    <td>Positive if the point is above the line, negative if below, zero if on the line.[web:1][web:3]</td>
  </tr>
  <tr>
    <td>Number of residuals</td>
    <td>Each observation in the dataset has one residual.[web:1][web:3]</td>
  </tr>
  <tr>
    <td>Sum and mean</td>
    <td>For an ordinary least squares regression, the sum and mean of residuals are 0.[web:1][web:3]</td>
  </tr>
  <tr>
    <td>Role in model fit</td>
    <td>Models are often chosen to minimize the sum of squared residuals; this is tied to R-squared and overall fit quality.[web:5][web:9]</td>
  </tr>
  <tr>
    <td>Diagnostic use</td>
    <td>Residual plots help detect nonlinearity, heteroscedasticity, and outliers; a random scatter around 0 suggests a good model.[web:4][web:5][web:7]</td>
  </tr>
</table>

Mini TL;DR

Residuals in linear regression are the per-point errors e=y−y^e=y-\hat{y}e=y−y^​ that show how far reality is from your model’s prediction; examining their size and pattern is key to judging and improving the model.

Information gathered from public forums or data available on the internet and portrayed here.