Interpreting a residual
A fitted line predicted one value and the real value came out different. How far off was the line, on which side, and what does that gap say about the thing measured?
Where you are 3 in Residuals
When you reach for it
A walking group timed itself on eight hiking trails, from 2 to 9 km long (made-up data, listed in full in Example 3). Technology fits the regression line
where x is the trail length in kilometers and \hat{y} is the predicted walking time in minutes. The 6 km trail took 103 minutes, but the line predicts 14(6) + 10 = 94. The walk took 9 minutes more than the line said. Is that a big miss? Which trail does the line get most wrong?
The gap between what happened and what the line predicted is the residual. You reach for it whenever a fitted line meets real data: to say how far off one prediction was, on which side, and which points the line fits worst. Getting the line belongs to Fitting linear and exponential models with technology, using it to predict to Predicting with a regression line, and extrapolation, and reading its slope to Interpreting the slope and intercept. A Line of best fit was already judged by its vertical gaps; this lesson names those gaps and measures them.
The procedure
Actual minus predicted. For a data point (x, y), put x into the line to get the predicted value \hat{y}, read “y-hat”. The hat marks a prediction, not a measurement. Then
The residual is in the units of y: minutes for the trails, dollars for a price.
The sign says the side. The order is fixed, actual first, so the sign carries meaning. A positive residual means the actual value was more than predicted: the point lies above the line and the line underestimated. A negative residual means less than predicted: the point lies below the line and the line overestimated. A residual of 0 means the point is on the line.
One sentence in context. A good interpretation names the individual, the size with units, the direction, and the comparison: “The walk on the 6 km trail took 9 minutes longer than the line predicts for a trail of that length.” The last words matter. The prediction depends on x, so a residual compares this trail with the prediction for its own length, not with the trails as a whole.
On the graph, the residual is the vertical gap. The predicted point (x, \hat{y}) is on the line straight above or below the data point (x, y), since both have the same x. So the residual is the vertical segment between them, counted upwards as positive and downwards as negative.
Why vertical, and not the shortest distance. The shortest path from a point to a line runs at a right angle to the line, on a slant. But the line predicts y from x: the trail length is known, and only the time is guessed. The miss is in y alone, so it is measured in the y direction, in the units of y. A slanted distance would mix kilometers with minutes, and its size would change if the times were written in hours. The vertical gap simply converts: 9 minutes is 0.15 hours.
The least-squares line. The line technology gives is the least-squares line: of all straight lines, it makes the sum of the squared residuals as small as possible. Squaring stops positive and negative residuals from canceling, and makes a big miss count heavily. One consequence, stated here without proof: the residuals of the least-squares line add to exactly 0, so the misses above the line balance the misses below it. Technology usually rounds its equation, and then the residuals add to almost 0; the made-up data in this lesson were chosen so the lines come out exact.
Size, and the worst fit. The size of a residual is its absolute value, so -4 is a bigger miss than 2. The point whose residual has the largest size is where the line fits worst.
Working backwards. Since residual = y - \hat{y}, adding \hat{y} to both sides gives
so any two of actual, predicted and residual give the third.
Worked examples
A walk that took longer than predicted
The 8 km trail took 124 minutes. Using \hat{y} = 14x + 10, find and interpret the residual.
SolutionFirst the prediction, by putting x = 8 into the line: \hat{y} = 14(8) + 10 = 122 minutes. Then actual minus predicted: 124 - 122 = 2 minutes.
The residual is positive, so the point lies above the line. The walk on the 8 km trail took 2 minutes longer than the line predicts for a trail of that length.
A book cheaper than predicted
A shop sells six paperbacks (made-up data, listed in full in Example 4). For page count x and price y in dollars, technology gives \hat{y} = 0.03x + 3.99. Find and interpret the residual for the 350-page book.
SolutionThe prediction is \hat{y} = 0.03(350) + 3.99 = 10.50 + 3.99 = 14.49 dollars. Actual minus predicted is 12.99 - 14.49 = -1.50 dollars.
The residual is negative, so the point lies below the line and the line overestimated. The 350-page book costs 1.50 dollars less than the line predicts for a book of that length.
Every residual on the trails
For all eight trails, find the residual, check that the residuals add to 0, and say which trail the line fits worst.
SolutionEach row puts x into \hat{y} = 14x + 10, then subtracts.
| x (km) | y (min) | \hat{y} | y - \hat{y} |
|---|---|---|---|
| 2 | 40 | 38 | 2 |
| 3 | 49 | 52 | -3 |
| 4 | 65 | 66 | -1 |
| 5 | 76 | 80 | -4 |
| 6 | 103 | 94 | 9 |
| 7 | 106 | 108 | -2 |
| 8 | 124 | 122 | 2 |
| 9 | 133 | 136 | -3 |
The sum is 2 - 3 - 1 - 4 + 9 - 2 + 2 - 3 = 0, as it must be for the least-squares line. The largest size is 9, at 6 km, so the line fits that trail worst: the walk took 9 minutes longer than predicted. Perhaps that trail is steeper, but the data cannot say why. The best fit is the 4 km trail, residual -1. In the scatter plot these are the longest and shortest segments.
The largest residual can be negative
For all six paperbacks, find the residuals, check their sum, and find the book the line fits worst.
SolutionEach prediction comes from \hat{y} = 0.03x + 3.99.
| x (pages) | y | \hat{y} | y - \hat{y} |
|---|---|---|---|
| 150 | 9.49 | 8.49 | 1.00 |
| 200 | 8.99 | 9.99 | -1.00 |
| 300 | 14.99 | 12.99 | 2.00 |
| 350 | 12.99 | 14.49 | -1.50 |
| 450 | 14.99 | 17.49 | -2.50 |
| 550 | 22.49 | 20.49 | 2.00 |
The sum is 1.00 - 1.00 + 2.00 - 1.50 - 2.50 + 2.00 = 0. The sizes are 1.00, 1.00, 2.00, 1.50, 2.50 and 2.00, so the worst fit is the 450-page book. Its residual is the smallest number in the column, yet it is the biggest miss: that book costs 2.50 dollars less than the line predicts for its length.
Working back to an actual value
(a) Suppose the time for the 7 km trail had been smudged, but technology’s list of residuals shows -2 for that trail. How long did that walk take? (b) One paperback’s page count is missing. Its price is 22.49 dollars and its residual is 2.00 dollars. How many pages does it have?
Solution(a) The prediction is \hat{y} = 14(7) + 10 = 108. Actual is predicted plus residual: y = 108 + (-2) = 106 minutes, which matches the table.
(b) This time the actual value and the residual are known, so the prediction is 22.49 - 2.00 = 20.49. The line gives that prediction where
Subtracting 3.99 from both sides gives 0.03x = 16.50, and dividing by 0.03 gives x = 550 pages.
Measuring a line drawn by eye
A line drawn by eye for the paperbacks, \hat{y} = 0.04x + 0.99, passes exactly through the 200-page book. Compare it with the technology line using residuals.
SolutionIts residuals are actual minus its predictions:
| x | y | by-eye \hat{y} | y - \hat{y} |
|---|---|---|---|
| 150 | 9.49 | 6.99 | 2.50 |
| 200 | 8.99 | 8.99 | 0 |
| 300 | 14.99 | 12.99 | 2.00 |
| 350 | 12.99 | 14.99 | -2.00 |
| 450 | 14.99 | 18.99 | -4.00 |
| 550 | 22.49 | 22.99 | -0.50 |
They add to -2.00, not 0, so its misses do not balance. Its squared residuals add to 6.25 + 0 + 4 + 4 + 16 + 0.25 = 30.50. For the technology line, from the last example, 1 + 1 + 4 + 2.25 + 6.25 + 4 = 18.50, which is smaller, as least squares promises. Fitting one book exactly cost the by-eye line a miss of 4.00 dollars at 450 pages.
Boundaries
- A residual belongs to one data point and one line. A different line gives different residuals, and adding a point to the data changes the least-squares line itself.
- A residual needs an actual value. For a value of x with no data, the line gives a prediction but no residual.
- Only the least-squares line, with its exact coefficients, gives the smallest possible total of squared residuals; its residuals add to 0, and to only nearly 0 once its coefficients are rounded. Other lines can have residuals adding to 0 too, so a sum of 0 alone does not make a line least squares (Exercise 11).
- Vertical gaps belong to predicting y from x. Predicting x from y would use horizontal gaps and usually give a different line.
- Sizes compare within one data set in the same units. A miss of 9 minutes cannot be set against a miss of 2.50 dollars.
- Small residuals do not prove a straight line is the right shape for the data.
Where this goes next
Making and reading residual plots plots each residual against x, laying the line flat so that a pattern the line missed, such as a curve, shows up. The squared residuals are exactly what Least squares regression makes as small as possible.
Exercises
Residuals on the club’s trails
Fabiola’s walking club timed six trails (made-up data). Technology fitted the line \hat{y} = 12x + 20 (time in minutes) to all six times and listed a residual of 7 for the 10 km trail. Later that time was smudged on the club’s sheet.
| length x (km) | time y (min) |
|---|---|
| 2 | 41 |
| 4 | 74 |
| 5 | 79 |
| 7 | 99 |
| 10 | ? |
| 12 | 160 |
- How long did the walk on the 10 km trail take?
- Find the residual for the 2 km trail and write a sentence interpreting it.
- Find the residual for the 4 km trail and write a sentence interpreting it.
- Find the residuals for the 5, 7 and 12 km trails, and check that all six add to 0.
- Which trail does the line fit worst, and which best? Is the worst one above or below the line?
Residuals on hardback prices
Another shop’s five hardbacks (made-up data) give the line \hat{y} = 0.04x + 7.99, price in dollars.
| pages x | price y (dollars) |
|---|---|
| 200 | 16.99 |
| 250 | 16.99 |
| 400 | 25.49 |
| 500 | 24.49 |
| 600 | 33.99 |
- Find the residual for the 400-page book and write a sentence interpreting it.
- Find the residual for the 500-page book. Is that point above or below the line?
- Find all five residuals, check their sum, and name the book the line fits worst.
- Two hardbacks cost 16.99 dollars. Using only the line, find the page count of the one whose residual is -1.00 dollars.
Other lines on the same data
- Move the club’s line up 2, to \hat{y} = 12x + 22. Using 147 minutes for the 10 km trail, find the moved line’s six residuals and their sum, then the sum of squared residuals for the moved line and for \hat{y} = 12x + 20.
- Predict every one of the eight trails in the worked examples with the flat line \hat{y} = 87, the mean time. Show that its residuals add to 0, and compare its sum of squared residuals with that of \hat{y} = 14x + 10.
- The club writes its times in hours instead, so its line becomes \displaystyle \hat{y} = 0.2x + \frac{1}{3}. Find the residual for the 10 km trail in hours, and compare it with Exercise 1.
Reasoning
- A residual is positive. Did the line overestimate or underestimate? And if it is negative?
- What does a residual of exactly 0 say about the point? Does it mean the line fits the data well?
- Explain why a residual is measured vertically and not along the shortest path to the line.
- Explain why the residuals of a least-squares line cannot all be positive.
- Exercise 11 gives a line whose residuals add to 0 but which fits badly. Explain why least squares adds up squared residuals rather than the residuals themselves.
Answers
- 147 minutes. The prediction is \hat{y} = 12(10) + 20 = 140, and y = 140 + 7 = 147.
- -3 minutes, since \hat{y} = 12(2) + 20 = 44 and 41 - 44 = -3. The walk on the 2 km trail took 3 minutes less than the line predicts for a trail of that length.
- 6 minutes, since \hat{y} = 12(4) + 20 = 68 and 74 - 68 = 6. The walk on the 4 km trail took 6 minutes longer than the line predicts for a trail of that length.
- 5 km: 79 - 80 = -1. 7 km: 99 - 104 = -5. 12 km: 160 - 164 = -4. With the others, -3 + 6 - 1 - 5 + 7 - 4 = 0.
- Worst: the 10 km trail, residual 7, the largest size. It is above the line: that walk took 7 minutes longer than predicted. Best: the 5 km trail, residual -1.
- 1.50 dollars, since \hat{y} = 0.04(400) + 7.99 = 23.99 and 25.49 - 23.99 = 1.50. The 400-page hardback costs 1.50 dollars more than the line predicts for a book of that length.
- -3.50 dollars, since \hat{y} = 0.04(500) + 7.99 = 27.99 and 24.49 - 27.99 = -3.50. The point is below the line.
- The predictions are 15.99, 17.99, 23.99, 27.99 and 31.99, so the residuals are 1.00, -1.00, 1.50, -3.50 and 2.00, which add to 0. The worst fit is the 500-page book, size 3.50.
- 250 pages. The prediction is 16.99 - (-1.00) = 17.99, so 0.04x + 7.99 = 17.99, giving 0.04x = 10 and x = 250.
- Each prediction of the moved line is 2 more, so each residual is 2 less: -5, 4, -3, -7, 5, -6, with sum -12. Its squares add to 25 + 16 + 9 + 49 + 25 + 36 = 160; the technology line’s add to 9 + 36 + 1 + 25 + 49 + 16 = 136, which is smaller.
- The flat line’s residuals y - 87 are -47, -38, -22, -11, 16, 19, 37 and 46, which add to 0. Their squares add to 8{,}360, against 4 + 9 + 1 + 16 + 81 + 4 + 4 + 9 = 128 for the technology line.
- \displaystyle \frac{7}{60} \approx 0.12 hours. The time is \displaystyle \frac{147}{60} = 2.45 hours and \displaystyle \hat{y} = 0.2(10) + \frac{1}{3} = \frac{7}{3}, so the residual is \displaystyle 2.45 - \frac{7}{3} = \frac{7}{60} hours, which is 7 minutes, as before. The units change the number, not the miss.
- Positive: the actual value is more than predicted, so the line underestimated. Negative: the line overestimated.
- The point is on the line, so the prediction for that x was exactly right. It says nothing about the other points: the line drawn by eye in Example 6 passes through one book and misses another by 4.00 dollars.
- The line predicts y from x, so x is taken as given and the prediction is the point on the line at the same x, directly above or below. The miss is in y alone and in the units of y. A slanted distance mixes the units of x and y, and changes size when a unit is changed.
- Its residuals add to 0. If every one were positive, the sum would be positive. So unless every point is on the line, at least one point lies below it.
- Plain residuals cancel: big misses above and below the line can add to 0, as for the flat line in Exercise 11. A square is never negative, so squares cannot cancel, their sum is 0 only when every point is on the line, and a big miss counts heavily.