Algebra 1 · S.ID.B.6B

Making and reading residual plots

A line has been fitted to some data and the dots sit close to it. How can you tell whether a straight line was the right shape to fit at all?

You need

Interpreting a residualAlgebra 1

Unlocks

Conditions for regression inferenceStatistics Transforming to linearity: logs and exponential modelsStatistics r squared and the standard deviation of the residualsStatistics

Where you are 3 in Residuals

Statistics7

Bivariate data9

Residuals3

Interpreting a residual

Making and reading residual plots

r squared and the standard deviation of the residuals

When you reach for it

Technology has given you the least-squares line for a scatter plot (the line of best fit it computes), and you want to know whether a line was a fair choice. The scatter plot alone often cannot tell you: when the dots climb across a wide range, a gentle bend hides inside them. A residual is actual minus predicted, y - \hat{y}, positive when the dot is above the line and negative when it is below.

A residual plot puts one dot at (x,\ y - \hat{y}) for every case, with a horizontal line at 0. It is the scatter plot with the fitted line laid flat and the gaps stretched to fill the picture, so what the line missed is the only thing left to see. Reach for it whenever you must decide whether a line describes the data or whether to choose another family. A correlation near 1 or -1 does not settle that: the correlation coefficient measures how tightly the dots hug a line, and Example 2 has r close to 1 and a curve all the same.

The procedure

Making one. For each case, put its x into the line to get the prediction \hat{y}, then subtract: the residual is y - \hat{y}. A table with columns x, y, \hat{y} and y - \hat{y} keeps each actual value next to its prediction. Then plot each pair (x,\ y - \hat{y}), with x across as before and residuals up the side, and draw the horizontal line at residual 0. That line stands for the fitted line itself, since a dot exactly on the fitted line has residual 0.

One check comes free. The residuals from a least-squares line always add to 0: the line sits so that the amounts above and below it balance. If yours add to something clearly not 0 (allowing a little for rounded coefficients), a prediction has been miscalculated. The check belongs to the least-squares line only; a line drawn by eye, or a curve fitted another way, need not balance.

Reading one. Look at the shape the dots make.

No pattern: dots above and below 0, mixed along the whole range, about equally spread. The line has caught the trend and what remains looks like chance. A line is a fair model.
A curve: a U (positive at both ends, negative in the middle) or an arch (the reverse). The data bend and the line cuts across the bend: a U means the data bend upward away from the line, an arch that they bend downward, as when growth levels off. A line is the wrong shape, however straight the scatter plot looked.
A fan: residuals spreading wider as x grows. The line follows the center, but predictions are less sure for larger x, where actual values stray further from it.
One dot far from the rest: that case does not follow the pattern. Check it for a slip. It may also have pulled the line toward itself, which shows as the other residuals leaning to one side.

To compare two models, make a residual plot for each and prefer the one that leaves no pattern.

Worked examples

Example 1

Making a residual plot: lengths of cord

Fulvio cuts six lengths of cord and weighs each (made-up data), giving pairs (length in meters, mass in grams): (2, 93), (4, 166), (6, 252), (8, 329), (10, 407), (12, 493). Technology gives the least-squares line \hat{y} = 40x + 10. Make the residual plot and say whether a line is a fair model.

Solution

Each prediction comes from putting the length into the line. For x = 2, \hat{y} = 40(2) + 10 = 90, so the residual is 93 - 90 = 3. The same for every cord:

x	y	\hat{y} = 40x + 10	y - \hat{y}
2	93	90	3
4	166	170	-4
6	252	250	2
8	329	330	-1
10	407	410	-3
12	493	490	3

The residuals add to 3 - 4 + 2 - 1 - 3 + 3 = 0, as they must. Plotting (2, 3), (4, -4) and the rest gives the lower panel.

Figure 1Fulvio’s cord (made-up data). The residual plot is the scatter plot with the line laid flat.

The dots sit above and below 0 with no bend and no widening. Every cord is within 4 grams of its prediction, small beside masses of 90 to 490 grams. A line is a fair model.

Example 2

A curve the scatter plot hides: pizza prices

Gertrud lists the price of a plain pizza at one shop by its diameter (made-up data), as pairs (diameter d in centimeters, price in dollars): (20, 6.75), (25, 8.25), (30, 9.5), (35, 12), (40, 14.5), (45, 18.25), (50, 21.75). Technology gives \hat{y} = 0.5d - 4.5. Make the residual plot and decide whether a line is a fair model.

Solution

Predict, then subtract. For d = 20, \hat{y} = 0.5(20) - 4.5 = 5.5 and the residual is 6.75 - 5.5 = 1.25.

d	y	\hat{y}	y - \hat{y}
20	6.75	5.5	1.25
25	8.25	8	0.25
30	9.5	10.5	-1
35	12	13	-1
40	14.5	15.5	-1
45	18.25	18	0.25
50	21.75	20.5	1.25

They add to 0.

Figure 2Gertrud’s pizza prices (made-up data). The scatter plot looks straight; the residual plot shows a U.

In the scatter plot every dot is within 1.25 dollars of the line on a scale running past 20, and the correlation is r \approx 0.98. The residual plot stretches those small gaps over its whole height, and a U appears: above at both ends, below through the middle. The prices curve upward more steeply than any line can. That matches how pizza is sold: you pay for the amount of pizza, its area, which grows with the square of the diameter. The line under-predicts the smallest and largest pizzas and over-predicts the middle ones, so a curved model would serve better.

Example 3

A fan: seeds that sprout

Gaia sows trays of seeds, two trays at each count, and records how many sprout (made-up data), as pairs (sown, sprouted): (10, 9), (10, 7), (20, 18), (20, 14), (30, 27), (30, 21), (40, 37), (40, 27), (50, 46), (50, 34). Technology gives \hat{y} = 0.8x. Make the residual plot and say what it means for predictions.

Solution

The line predicts 0.8 \times 10 = 8 sprouts from 10 seeds, and so on.

x	\hat{y}	y	y - \hat{y}
10	8	9, 7	1, -1
20	16	18, 14	2, -2
30	24	27, 21	3, -3
40	32	37, 27	5, -5
50	40	46, 34	6, -6

Figure 3Gaia’s seed trays (made-up data). The residuals fan out as more seeds are sown.

There is no bend: at every count the residuals balance about 0, so the line follows the center of the data. But the spread widens from \pm 1 at 10 seeds to \pm 6 at 50, a fan. A prediction for 10 seeds is likely within about 1 sprout; one for 50 seeds could be 6 out.

Example 4

One point far from the rest: song files

Gottfried notes the length and file size of seven songs (made-up data), as pairs (minutes, megabytes): (2, 1.5), (3, 3), (4, 7.5), (5, 4.75), (6, 6), (7, 7.25), (8, 8.5). Technology gives \hat{y} = x + 0.5. Make the residual plot and read it.

Solution

Predict, then subtract:

x	y	\hat{y}	y - \hat{y}
2	1.5	2.5	-1
3	3	3.5	-0.5
4	7.5	4.5	3
5	4.75	5.5	-0.75
6	6	6.5	-0.5
7	7.25	7.5	-0.25
8	8.5	8.5	0

The residuals add to 0.

Figure 4Gottfried’s song files (made-up data). One song stands far from the rest.

One dot, the 4-minute song at 3, stands far from the rest, which all lie between -1 and 0. Every other residual is 0 or negative: that one song has pulled the line up toward itself and left the others below it. Gottfried should check that song. If it was saved at a higher quality setting, it does not belong with the others and the line should be refitted without it; if the size was written down wrongly, it should be corrected.

Example 5

Line or exponential: water fern on a pond

Gabor measures the area of a pond covered by water fern, y square meters, at the start of each week x (made-up data): (0, 1.9), (1, 3.1), (2, 4.5), (3, 6.9), (4, 10), (5, 15.2), (6, 22.8). Technology fits the line \hat{y} = 3.3x - 0.7 and the exponential model \hat{y} = 1.981(1.504)^x. Use residual plots to choose between them.

Solution

Work out both sets of residuals. For the line:

x	y	\hat{y} = 3.3x - 0.7	y - \hat{y}
0	1.9	-0.7	2.6
1	3.1	2.6	0.5
2	4.5	5.9	-1.4
3	6.9	9.2	-2.3
4	10	12.5	-2.5
5	15.2	15.8	-0.6
6	22.8	19.1	3.7

For the exponential, to two decimal places:

x	y	\hat{y} = 1.981(1.504)^x	y - \hat{y}
0	1.9	1.98	-0.08
1	3.1	2.98	0.12
2	4.5	4.48	0.02
3	6.9	6.74	0.16
4	10	10.14	-0.14
5	15.2	15.24	-0.04
6	22.8	22.93	-0.13

Figure 5Gabor’s water fern (made-up data). The line leaves a U; the exponential leaves almost nothing.

The line’s residuals add to 0 and form a clear U: the fern grows faster and faster, and the line cannot bend with it. The exponential’s residuals are all within 0.16 of 0, with signs mixed and no shape. (They add to -0.09, not 0: the balance belongs to least-squares lines, and this curve is not one.) The exponential model fits: each week the area is multiplied by about 1.5. Fitting both models is covered in Fitting linear and exponential models with technology.

Boundaries

A residual plot judges a model only over the x values in the data. No pattern there says nothing about x values beyond them; see Predicting with a regression line, and extrapolation.
With few points a pattern can appear by chance, and a real one can hide. Four or five dots give a weak verdict.
The residuals of a least-squares line add to exactly 0 when its coefficients are unrounded; rounded coefficients leave a sum near 0. Other models need not balance at all (Example 5).
No pattern means a line describes the data fairly. It does not mean x causes y, and it does not mean predictions are close: the size of the residuals, read off the vertical scale, says how close.
A curved pattern says a line is the wrong shape, not which curve is right. Compare the residual plots of the candidate models, as in Example 5. Re-expressing data so that a curve becomes straight is Transforming to linearity: logs and exponential models.

Where this goes next

The residual plot is the working check behind every regression decision that follows. The typical size of the residuals, and the share of the variation a line accounts for, become numbers of their own in r squared and the standard deviation of the residuals. Before drawing conclusions about a whole population from a sample, Conditions for regression inference reads the same plot for a straight trend (no curve) and an even spread (no fan). And when a residual plot shows a curve, the next move is to fit another family with technology, or to straighten the data with a transformation.

Exercises

Making residual plots

Finnian times five bike rides (made-up data), as pairs (kilometers, minutes): (5, 18), (10, 36), (15, 49), (20, 62), (25, 80). Technology gives \hat{y} = 3x + 4. Find each residual and check that they add to 0.
Sketch the residual plot for Exercise 1 and decide whether a line is a fair model.
Six residuals from a least-squares line are 1.2, -0.7, 0.4, -1.5, 0.9, and one more that has been smudged. What is the missing residual?

Matching scatter plots to residual plots

Figure 6Scatter plots A to D, and the residual plots of their least-squares lines in a shuffled order (made-up data).

Which residual plot belongs to scatter plot A?
Which belongs to scatter plot B?
Which belongs to scatter plot C?
Which belongs to scatter plot D?

Reading residual plots

Fulvio charges a phone and records the battery percentage at 10, 20, \dots, 70 minutes (made-up data). The residuals from the least-squares line are, in order, -5, 0, 3, 4, 3, 0, -5. Describe the pattern and what it says about how the battery fills.
For a restaurant, a least-squares line predicts the bill from the size of the party (made-up data). Its residuals, in dollars, are 3 and -3 for parties of 2; 6 and -6 for parties of 4; 10 and -10 for 6; 15 and -15 for 8. Describe the pattern. Is a prediction surer for a party of 2 or of 8?
Use Gaia’s line in Example 3 to predict the sprouts from 25 seeds, and use the residual plot to say roughly how far off the prediction might be.
A market stall’s least-squares line for six days has residuals 1, 0.5, 1, -6, 2 and 1.5 (made-up data, days 1 to 6). Which day stands apart, and why are all the other residuals positive?
Technology fits a line and an exponential model to the number of fruit flies in a jar over 8 days (made-up data). The line’s residuals are 24, 6, -9, -21, -21, -9, 6, 24; the exponential’s are 2, -3, 1, 4, -2, -1, 3, -4. Which model should be used, and why?
Use Gertrud’s line in Example 2 to predict the price of a 60 cm pizza. Is the true price more likely above or below the prediction? Give two reasons to be wary of it.

Explaining

Can every residual from a least-squares line be positive? Explain.
A residual plot has dots both above and below 0. Explain why that alone does not show that a line fits.
Every dot of a scatter plot lies exactly on the least-squares line. Describe the residual plot.
A friend finds r \approx 0.98 for some data and says that proves a line is the right model. Use Example 2 to answer.
Explain why a residual plot shows a bend more clearly than the scatter plot does.

Answers

Residuals -1, 2, 0, -2, 1. The predictions are 19, 34, 49, 64, 79, and -1 + 2 + 0 - 2 + 1 = 0.
Dots at (5, -1), (10, 2), (15, 0), (20, -2), (25, 1) lie above and below 0 with no bend or widening. A line is a fair model; every ride is within 2 minutes of its prediction.
-0.3. The five known residuals add to 0.3, and all six must add to 0.
Plot 2. A is a straight band, so nothing but scatter is left.
Plot 4. B curves upward away from its line, so the residuals form a U.
Plot 1. C levels off, so the line runs above the ends and below the middle: an arch.
Plot 3. D’s band widens to the right, so the residuals fan out.
An arch: negative at both ends, positive in the middle. The percentage rises quickly at first and then more slowly, leveling off, so a line is the wrong shape.
A fan: the spread grows with party size. A prediction for 2 people is likely within about 3 dollars; one for 8 could be 15 dollars out, so the party of 2 is surer.
\hat{y} = 0.8 \times 25 = 20 sprouts. The residuals near 25 seeds are \pm 2 and \pm 3, so expect roughly 17 to 23.
Day 4, at -6. The residuals add to 0, so that one low day has pulled the line down toward itself, leaving the other five above it. Check day 4 and consider refitting without it.
The exponential. The line’s residuals form a U, so the flies multiply faster than a line allows; the exponential’s are small with mixed signs and no shape.
\hat{y} = 0.5(60) - 4.5 = 25.5 dollars. Likely above: the residuals are positive and rising at the right end of the U. Be wary because the line is the wrong shape, and because 60 cm is outside the data.
No. They add to 0, so if any is positive another must be negative, unless all are 0.
A U or an arch also has dots on both sides (Example 2). What matters is the shape: whether positive and negative residuals are mixed along the whole range or grouped.
Every residual is 0, so every dot lies on the line at 0.
Example 2 has r \approx 0.98 and still shows a U. A high r says the dots hug a line tightly; a gentle curve can do that. Only the residual plot shows whether the shape is straight.
Subtracting \hat{y} moves each dot by the height of the line at its x, laying the line flat and removing the trend. Only the gaps are left, drawn on a scale of a few units instead of the whole range of y, so a bend of 1 unit in 20 fills the picture.