Residuals in a Linear Model

A linear model uses an equation such as ŷ = a + bx to predict a response from an explanatory variable. The predicted value is ŷ (read 'y-hat'); the observed value is y. Their difference, y − ŷ, is the residual. It records the model's error for one observation while preserving direction: positive and negative errors mean the point falls on different sides of the fitted line.

Calculate a residual

Suppose a model predicts a travel time of 42 minutes for a trip, but the observed time is 47 minutes. The residual is 47 − 42 = 5 minutes. The actual time was five minutes greater than predicted, so the residual is positive. If the actual time had been 39 minutes, the residual would be 39 − 42 = −3 minutes: the model overpredicted by three minutes.

Always subtract predicted from observed, in that order. Reversing the subtraction changes the sign and the interpretation. Use the prediction from the model for the same x-value as the observed point, and keep units consistent. If y is measured in dollars, the residual is in dollars; if y is measured in minutes, the residual is in minutes.

Read the residual on a scatterplot

On a scatterplot with a fitted line, a residual is the vertical distance from an observed point to the line at the same x-value. A point above the line has y greater than ŷ and a positive residual. A point below has y less than ŷ and a negative residual. A point on the line has residual zero.

The residual is vertical rather than the shortest geometric distance to the line because the model predicts y from x. It compares actual and predicted response values at a fixed explanatory value. Horizontal distance would answer a different question.

Use residuals to assess fit

A small absolute residual means the model's prediction is close for that observation; a large absolute residual means the prediction misses by more. Looking across all residuals helps evaluate whether a straight-line model captures the pattern. If residuals are scattered around zero without a visible structure, a linear model may be reasonable for the observed range.

A curved pattern in a residual plot suggests that the relationship is not well represented by a straight line. A fan shape, where residual spread grows as x increases, suggests that prediction errors vary across the range. A cluster of unusually large residuals may identify an outlier or an observation worth checking. These patterns are diagnostic clues, not automatic proof of a particular cause.

For example, residuals of 2, −1, and 3 points for three students mean the model underpredicted the first by 2, overpredicted the second by 1, and underpredicted the third by 3. The signs help explain direction, while the absolute values give the miss size. If a question asks which prediction was closest, compare absolute residuals: |−1| is smaller than |2| and |3|, so the second prediction was closest even though its residual is negative.

A residual plot uses x-values on the horizontal axis and residuals on the vertical axis, with a reference line at zero. Points should be scattered on both sides of zero without a curve or changing spread for a simple linear model to be a reasonable fit. A positive residual cluster at small x and negative cluster at large x may reveal a systematic overprediction or underprediction pattern that a single overall average can hide.

Residuals and outliers

An observation with a large residual is poorly predicted by the model. It may be an outlier in the y-direction, but residual size alone does not determine how influential the observation is on the fitted line. A point far from the center of the x-values can have substantial influence even when its residual is not the largest. For a basic interpretation question, distinguish 'unusual vertical error' from 'strong effect on the line.'

Example using an equation

A model for quiz score is ŷ = 12 + 4x, where x is hours studied. A student studies 5 hours and earns 34 points. The predicted score is 12 + 4(5) = 32. The residual is 34 − 32 = 2 points. The student scored two points above the model's prediction. Another student who earns 29 points after studying 5 hours has residual 29 − 32 = −3 points, so the model predicted three points too high for that student.

Residuals are not percentages unless the problem defines a relative error. A residual of 2 points does not by itself mean the prediction was 2% low. To calculate a percent error, divide an appropriate absolute difference by a stated reference value and multiply by 100%; that is a different measure.

The average residual from an ordinary least-squares line with an intercept is zero for the data used to fit it, because positive and negative errors balance. That does not mean the predictions are perfect: large positive and negative residuals may cancel. Inspect their spread and pattern as well as their average. For new observations or a line supplied by a question, the residuals need not average to zero.

Common mistakes

• Subtracting observed from predicted and reversing the sign convention.
• Calling the residual the predicted value itself instead of the prediction error.
• Treating a positive residual as an overprediction; positive means observed exceeds predicted.
• Using horizontal distance from a point to the line.
• Assuming every large residual proves the observation is wrong.
• Interpreting residuals outside the range of observed x-values as reliable forecasts.

Exam takeaway

Calculate e = y − ŷ using observed minus predicted. Positive means the point lies above the line and actual y is larger; negative means it lies below and actual y is smaller. Use residual size and patterns to describe fit, while keeping the model's range and the context in view.

Calculate a residual from a prediction

Residual = observed value − predicted value. If a model predicts 68 points and the actual score is 73, the residual is 73 − 68 = 5. The positive residual means the observation sits five units above the model’s prediction. If the actual score were 63, the residual would be −5 and the point would sit below the predicted value.

Use the fitted line to get the prediction

Suppose a fitted model is ŷ = 12 + 3x. At x = 4, the predicted response is 12 + 3(4) = 24. If the observed response is 21, the residual is 21 − 24 = −3. Keep the predicted value separate from the observed value: reversing the subtraction changes the interpretation and sign.

Read a residual plot

A residual plot displays residuals against the explanatory variable or fitted values, with a horizontal reference line at zero. A roughly random cloud above and below zero is consistent with a linear model being useful for the range shown. A curved pattern suggests the relationship may bend; a fan shape suggests the spread changes with x; clusters may indicate groups that behave differently. A single large residual can flag an unusual observation worth checking.

Understand what residuals do not prove

Residuals describe errors for the observations used to fit or assess the model. Small residuals do not establish that x causes y, and an apparently random residual plot does not guarantee accurate predictions far outside the observed range. A model can fit a sample while missing a relevant variable, relying on biased data, or behaving poorly for a new population.

A residual of zero means the observed value equals the model prediction for that case. It does not mean the entire model is exact. When comparing residual magnitudes, consider the response’s units and the scale of the data; a residual of 4 may be minor in one context and substantial in another.