math.wiki
Statistics › Bivariate data › Residuals › r squared and the standard deviation of the residuals
Open on the map
Statistics · S.ID.C.8 · S.ID.B.6B

r squared and the standard deviation of the residuals

A least-squares line has been fitted. How much of the variation in the response does it actually account for, and how far off are its predictions, typically?

You need
Making and reading residual plotsAlgebra 1The correlation coefficientAlgebra 1
Unlocks
Reading regression outputStatistics
Where you are 3 in Residuals
Statistics7
Bivariate data9
Residuals3
Interpreting a residualMaking and reading residual plots
r squared and the standard deviation of the residuals

Why this exists

Hodan keeps eight beehives. At the end of the season she records, for each hive, the number of frames of drawn comb in its honey boxes and the honey she harvests, and fits a least-squares line. The line rises, and the correlation says the dots cluster fairly closely around it. But a beekeeper wants plainer answers. Of all the differences in harvest from hive to hive, how much does the number of frames account for? And when she uses the line to predict next year’s harvest, how far off should she expect to be?

Neither the slope nor r answers those questions directly. This lesson builds two numbers that do. The first, r squared, compares the line’s misses with the misses of the best guess made without x at all. The second, s, puts the size of a typical miss in pounds. The lesson takes two days: r squared on the first, s and how the two numbers differ on the second.

Definition

Without the explanatory variable, the best single prediction of every y is the mean \bar{y}. Its misses are the deviations y - \bar{y}, and the total sum of squares collects them:

\text{SST} = \sum (y - \bar{y})^2.

With the least-squares line, each miss is a residual y - \hat{y}, and the sum of squared errors, met in Judging and comparing models, collects those:

\text{SSE} = \sum (y - \hat{y})^2.

The coefficient of determination, written r^2 and read “r squared”, is the share of \text{SST} that the line removes:

r^2 = 1 - \frac{\text{SSE}}{\text{SST}} = \frac{\text{SST} - \text{SSE}}{\text{SST}}.

In words: r squared is the fraction of the variation in y that is accounted for by the least-squares line relating y to x. In context it is reported as a percent of the variation in the named response, accounted for by the line on the named explanatory variable.

The standard deviation of the residuals is

s = \sqrt{\frac{\text{SSE}}{n - 2}} = \sqrt{\frac{\sum (y - \hat{y})^2}{n - 2}},

where n is the number of points. It is in the units of y, and it measures the typical size of a prediction error: how far, typically, an actual value falls from the value the line predicts.

Intuition

0 2 4 6 8 10 0 10 20 30 40 50 60 frames of drawn comb honey (lb), misses from the mean 0 2 4 6 8 10 0 10 20 30 40 50 60 frames of drawn comb honey (lb), misses from the line
Figure 1Hodan’s hives (made-up data). Top, how far each harvest is from the mean; bottom, how far each is from the line. The squares of the top segments add to 1986, those of the bottom segments to 486.

The top panel shows the misses of the flat line at \bar{y}: each vertical segment is a deviation y - \bar{y}. The bottom panel shows the misses of the least-squares line: each segment is a residual. The segments shrink because the line uses the frames, and r squared measures how much they shrink, counted in squared units so that the two totals can be compared with each other. A line that misses as badly as \bar{y} does has r^2 = 0; a line through every dot has \text{SSE} = 0 and r^2 = 1.

s looks at the bottom panel alone. It is a kind of average length of those segments: square them, add, divide by n - 2, take the square root, the way a Standard deviation is built from deviations from a mean.

Results

For a least-squares line with one explanatory variable:

\text{SST} = \text{SSE} + \sum (\hat{y} - \bar{y})^2.
r^2 = (r)^2, \quad \text{so} \quad 0 \le r^2 \le 1 \quad \text{and} \quad r = \pm\sqrt{r^2},

where r takes the sign of the slope. And with s_y the standard deviation of y,

s = \sqrt{\frac{(1 - r^2)\,\text{SST}}{n - 2}} = s_y\sqrt{\frac{(1 - r^2)(n - 1)}{n - 2}}.
Why it is true

Two facts about the least-squares line carry the proof. It passes through (\bar{x}, \bar{y}) with slope b = r\dfrac{s_y}{s_x} (Slope from r and the spreads, and regression to the mean), so \hat{y} - \bar{y} = b(x - \bar{x}). And its residuals e = y - \hat{y} balance in two ways: \sum e = 0 and \sum x e = 0. Both come from the line having the smallest sum of squares (The least squares criterion and its line). Raise the whole line by a number c: each residual becomes e - c, and the sum of squares becomes \sum (e - c)^2 = \text{SSE} - 2c\sum e + nc^2. If \sum e were not 0, a small enough c with the same sign as \sum e would make this less than \text{SSE}, which cannot happen for the least-squares line. Tilt the line instead, adding c to its slope: each residual becomes e - cx, the sum of squares becomes \text{SSE} - 2c\sum xe + c^2\sum x^2, and the same argument forces \sum xe = 0. So also \sum (x - \bar{x})e = \sum xe - \bar{x}\sum e = 0.

Split each deviation as y - \bar{y} = e + (\hat{y} - \bar{y}), square and add:

\begin{aligned} \text{SST} &= \sum e^2 + 2\sum e(\hat{y} - \bar{y}) + \sum (\hat{y} - \bar{y})^2 \\ &= \text{SSE} + 2b\sum e(x - \bar{x}) + \sum (\hat{y} - \bar{y})^2. \end{aligned}

The middle sum is 0, which gives the first result. For the last sum,

\begin{aligned} \sum (\hat{y} - \bar{y})^2 &= b^2 \sum (x - \bar{x})^2 = r^2\frac{s_y^2}{s_x^2}(n - 1)s_x^2 \\ &= r^2 (n - 1)s_y^2 = r^2\,\text{SST}, \end{aligned}

since \text{SST} = (n - 1)s_y^2 by the definition of s_y. So \text{SSE} = \text{SST} - r^2\,\text{SST}, which rearranges to 1 - \dfrac{\text{SSE}}{\text{SST}} = r^2, the square of the correlation. Because \text{SSE} and \sum(\hat{y} - \bar{y})^2 are sums of squares, neither is negative, so 0 \le \text{SSE} \le \text{SST} and r^2 lies between 0 and 1. Substituting \text{SSE} = (1 - r^2)\text{SST} into the definition of s gives the last result.

Why n - 2 and not n - 1? A standard deviation divides by n - 1 because one number, \bar{x}, was estimated from the data first. A line costs two numbers, the slope and the intercept. With n = 2 points the line passes through both, every residual is 0, and the data say nothing about the spread around a line; dividing by n - 2 = 0 reports exactly that, as no answer.

Worked examples

Examples 1 to 3 belong to the first day (r squared), Examples 4 to 6 to the second (the standard deviation of the residuals, and how the two numbers differ).

Example 1

r squared from the two sums of squares

Hodan’s eight hives (made-up data), as pairs (frames of drawn comb, pounds of honey): (2, 8), (3, 14), (4, 34), (5, 16), (7, 45), (8, 46), (9, 51), (10, 42). Technology gives the least-squares line \hat{y} = 2 + 5x. Find \text{SST}, \text{SSE} and r squared, and interpret r squared in context.

Solution

The mean harvest is \bar{y} = \dfrac{256}{8} = 32 pounds. Each prediction comes from the line, for example \hat{y} = 2 + 5(2) = 12 for the first hive. One table holds the misses from the mean, a second the misses from the line:

xyy - \bar{y}(y - \bar{y})^2
28-24576
314-18324
43424
516-16256
74513169
84614196
95119361
104210100
xy\hat{y}y - \hat{y}(y - \hat{y})^2
2812-416
31417-39
4342212144
51627-11121
74537864
84642416
95147416
104252-10100

Adding the squared columns, \text{SST} = 1986 and \text{SSE} = 486 (squared pounds). So

r^2 = 1 - \frac{486}{1986} = \frac{1500}{1986} \approx 0.755.

About 76\% of the variation in honey harvested is accounted for by the least-squares line relating honey to the number of frames of drawn comb. The other 24\% is left in the residuals: differences between hives with the same number of frames, from causes the line does not include.

Example 2

The square of the correlation

For the same eight hives, find r from the data and confirm that its square is the r squared of Example 1. Then say what r squared does not tell Hodan.

Solution

By the formula from The correlation coefficient, with \sum (x - \bar{x})(y - \bar{y}) = 300, \sum (x - \bar{x})^2 = 60 and \sum (y - \bar{y})^2 = 1986,

r = \frac{300}{\sqrt{60 \times 1986}} \approx 0.869.

Then r^2 \approx 0.869^2 \approx 0.755, the same value as before, as the Results promise. Squaring loses the sign: the line rises, so r is positive, but r squared alone would read 0.755 for a falling line too.

What r squared does not say: that frames cause the honey (a strong colony both builds comb and gathers nectar, so a third cause could drive both); that 76\% of hives are predicted correctly (it is a share of variation, not of hives); or that a straight line is the right shape, which only a residual plot can show.

Example 3

From r to r squared and back

(a) Warsame finds r = -0.62 between the number of mites per 100 bees and the honey harvested. Find and interpret r squared. (b) Faadumo reports r squared = 0.81 for a line predicting honey from a hive’s spring weight, and the line rises. Find r. (c) Is a correlation of 0.7 “a bit stronger” than one of 0.5?

Solution

(a) r^2 = (-0.62)^2 = 0.3844. About 38\% of the variation in honey harvested is accounted for by the least-squares line relating honey to mite count. The negative sign says harvests fall as mites rise; r squared measures how much, not which way.

(b) \displaystyle r = \pm\sqrt{0.81} = \pm 0.9, and the sign is the slope’s. The line rises, so r = 0.9. Without knowing the direction, both signs would be possible.

(c) 0.5^2 = 0.25 and 0.7^2 = 0.49. The second line accounts for 49\% of the variation in its response and the first for 25\%: nearly twice the share, from a correlation only 1.4 times as large. r squared is the scale on which shares of variation add up, which is why it is reported.

Example 4

The typical prediction error

Find the standard deviation of the residuals for Hodan’s line, interpret it, and compare it with the standard deviation of the harvests themselves.

Solution

From Example 1, \text{SSE} = 486 and n = 8, so

s = \sqrt{\frac{486}{8 - 2}} = \sqrt{81} = 9 \text{ pounds}.

When the line is used to predict a hive’s honey harvest from its frames of drawn comb, the prediction is typically off by about 9 pounds.

The harvests themselves have \displaystyle s_y = \sqrt{\dfrac{1986}{7}} \approx 16.8 pounds: guessing 32 pounds for every hive would typically miss by about 17 pounds. Using the frames cuts the typical miss roughly in half. The two numbers are linked by the Results: \displaystyle 9 = 16.8\sqrt{(1 - 0.755)\cdot\tfrac{7}{6}}, to rounding.

Example 5

Putting s to work on a prediction

Next season Hodan has a hive with 6 frames of drawn comb. Predict its harvest and use s to say what range of harvests would be ordinary.

Solution

\hat{y} = 2 + 5(6) = 32 pounds. A typical miss is about 9 pounds, so a harvest from about 23 to 41 pounds would be unremarkable. The figure shows the band 9 pounds either side of the line.

0 2 4 6 8 10 0 10 20 30 40 50 60 70 frames of drawn comb honey (lb)
Figure 2Hodan’s hives (made-up data) with the band from 9 pounds below the line to 9 pounds above it, one standard deviation of the residuals on each side.

Five of the eight hives lie inside it, and the largest residual is 12 pounds. So s is a typical miss, not a largest one: some hives fall outside the band, and a hive 18 pounds or more off the line (two typical misses) would be worth a closer look. The band is only fair because the residuals spread about evenly along the line; 6 frames also sits inside the data, from 2 to 10, so this is not a prediction outside the range.

Example 6

Two numbers, two questions

Two apiaries fit lines predicting honey (pounds) from frames. At the hilltop apiary, r squared = 0.90 and s = 12 pounds; at the valley apiary, r squared = 0.55 and s = 4 pounds. (a) Which line accounts for more of the variation in its harvests? (b) Which gives closer predictions? (c) The valley beekeeper converts to kilograms (1 pound = 0.4536 kilogram). What happens to r squared and to s?

Solution

(a) The hilltop line: 90\% of the variation in its harvests, against 55\%.

(b) The valley line: its predictions are typically off by about 4 pounds, against 12. Both answers hold at once because the two numbers ask different questions. r squared compares the line with the hilltop’s own mean, and the hilltop harvests vary enormously from hive to hive, so a line can remove most of that variation and still leave misses of 12 pounds. The valley harvests are all much alike, so even a weaker line misses by little. For “how much does x help?” read r squared; for “how close is a prediction?” read s.

(c) Every residual and every deviation is multiplied by 0.4536, so \text{SSE} and \text{SST} are both multiplied by 0.4536^2 and their ratio is unchanged: r squared stays 0.55. But s carries the units of y: 4 \times 0.4536 \approx 1.81 kilograms.

Boundaries

  • r^2 = (r)^2 holds for the least-squares line with one explanatory variable. For any other line, 1 - \dfrac{\text{SSE}}{\text{SST}} is smaller, and for a line worse than \bar{y} it is negative.
  • r squared says how much variation a line accounts for, not whether a line is the right shape. A curve can give a high r squared; read the residual plot first.
  • Neither number shows cause. “Accounted for” is a statement about the data, not about what drives y.
  • s describes a typical miss when the residuals spread evenly. If the residual plot fans out, predictions at one end are closer than s suggests and at the other end farther.
  • s needs at least 3 points: with n = 2 it divides by 0.
  • r squared needs the y values to vary. If every y is the same, \text{SST} = 0 and \dfrac{\text{SSE}}{\text{SST}} is undefined: there is no variation to account for.
  • Both numbers describe the data in hand. They say nothing about x values outside the data (Predicting with a regression line, and extrapolation), and one unusual point can change both a great deal (Outliers, leverage and influential points).

Where this goes next

Statistical software prints r squared and s beside the slope and intercept, and Reading regression output reads them off a printed table. Outliers, leverage and influential points watches how both numbers change when one point is removed, and Transforming to linearity: logs and exponential models straightens curved data so that a line, with its r squared and s, describes it fairly. Later, s becomes the estimate of the spread about the true line in inference for regression.

Exercises

Exercises 1 to 8 go with the first day (Examples 1 to 3), Exercises 9 to 14 with the second (Examples 4 to 6), and the last group draws on both.

Part 1: r squared

  1. For Faadumo’s hives, a line predicting honey from frames has \text{SST} = 800 and \text{SSE} = 200. Find r squared and interpret it in context.
  2. A line predicting pounds of beeswax from pounds of honey has r = 0.6. Find r squared and interpret it.
  3. A line predicting honey from the number of days of rain in summer has r = -0.95. Find r squared.
  4. A line predicting honey from the age of the queen has r squared = 0.49 and a negative slope. Find r.
  5. A report gives r squared = 0.64 and nothing else. What can r be?
  6. Warsame records how many weeks each of five hives spent near a clover field and its honey in pounds (made-up data): (1, 10), (2, 12), (3, 21), (4, 22), (5, 30). Technology gives \hat{y} = 4 + 5x. Find \text{SSE}, \text{SST} and r squared.
  7. For a line with r squared = 0.30, Faadumo says “the line predicts 30\% of the hives correctly.” Correct her statement.
  8. A line has r squared = 0.85 and \text{SST} = 2400. Find \text{SSE}.

Part 2: the standard deviation of the residuals

  1. A line fitted to 12 hives has \text{SSE} = 490, in squared pounds. Find s and interpret it.
  2. Find s for Warsame’s line in Exercise 6 and interpret it.
  3. A new hive has 5 frames and yields 40 pounds. Use Hodan’s line from Example 1 to find its residual, and compare it with her s.
  4. A line predicting honey in kilograms has r squared = 0.70 and s = 4 kilograms. Give r squared and s if the harvests are recorded in pounds instead (1 pound = 0.4536 kilogram).
  5. For the same 12 hives, \text{SST} = 1000. A line on frames has \text{SSE} = 400; a line on spring weight has \text{SSE} = 250. Find r squared and s for each. Which line is better?
  6. A line fitted to 14 hives has \text{SST} = 1500 and r squared = 0.8. Find s.

Explaining

  1. Explain why r squared for a least-squares line can never be negative or greater than 1.
  2. A least-squares line has r squared = 0. Describe the line.
  3. Explain why s divides by n - 2, and what happens with only two points.
  4. A line has r squared = 0.98. Does that show a straight line is the right model? Explain.
  5. Why does r squared have no units while s has the units of y?
  6. Hodan’s line has r squared \approx 0.76. A friend says “frames of drawn comb cause 76\% of the honey.” Explain what is wrong.
Answers
  1. r squared = 1 - \dfrac{200}{800} = 0.75. About 75\% of the variation in honey harvested is accounted for by the least-squares line relating honey to frames.
  2. r squared = 0.6^2 = 0.36. About 36\% of the variation in beeswax is accounted for by the least-squares line relating beeswax to honey.
  3. r squared = (-0.95)^2 = 0.9025, about 90\%.
  4. r = -0.7. The square roots of 0.49 are \pm 0.7, and r takes the sign of the slope.
  5. r = 0.8 or r = -0.8. Squaring removes the sign, so the direction must come from the slope or the scatter plot.
  6. \text{SSE} = 14, \text{SST} = 264, r squared \approx 0.947. The predictions are 9, 14, 19, 24, 29, so the residuals are 1, -2, 2, -2, 1. The mean is \bar{y} = 19 and the deviations are -9, -7, 2, 3, 11. Then 1 - \dfrac{14}{264} = \dfrac{250}{264}.
  7. r squared is a share of variation, not of hives: about 30\% of the variation in honey harvested is accounted for by the line, and 70\% is left in the residuals.
  8. \text{SSE} = 360. From \text{SSE} = (1 - r^2)\,\text{SST} = 0.15 \times 2400.
  9. \displaystyle s = \sqrt{\dfrac{490}{10}} = 7 pounds. Predictions from this line are typically off by about 7 pounds of honey.
  10. \displaystyle s = \sqrt{\dfrac{14}{3}} \approx 2.16 pounds. Predicting a hive’s honey from its weeks near the clover field is typically off by about 2.2 pounds.
  11. \hat{y} = 2 + 5(5) = 27, so the residual is 40 - 27 = 13 pounds: about 1.4 times s = 9. Larger than a typical miss, but well short of two typical misses.
  12. r squared stays 0.70; s = \dfrac{4}{0.4536} \approx 8.82 pounds. A change of units scales \text{SSE} and \text{SST} alike but scales s with y.
  13. Frames: r squared = 0.6, \displaystyle s = \sqrt{40} \approx 6.32 pounds. Spring weight: r squared = 0.75, s = 5 pounds. The spring-weight line is better on both counts; with the same response and the same hives, smaller \text{SSE} means both a larger r squared and a smaller s.
  14. s = 5 pounds. \text{SSE} = 0.2 \times 1500 = 300 and \displaystyle s = \sqrt{\dfrac{300}{12}} = 5.
  15. \text{SSE} is a sum of squares, so it is at least 0, which keeps r squared at most 1. The horizontal line at \bar{y} is one of the lines the least-squares line beats, and its \text{SSE} is \text{SST}, so \text{SSE} \le \text{SST} and r squared is at least 0.
  16. It is the horizontal line \hat{y} = \bar{y}. r squared = 0 means r = 0, so the slope b = r\dfrac{s_y}{s_x} is 0, and the line passes through (\bar{x}, \bar{y}).
  17. The slope and intercept are both estimated from the data, which uses up two of the n values. With two points the line passes through both, \text{SSE} = 0 and n - 2 = 0: the data hold no information about spread around a line, and s is undefined.
  18. No. A gently curved pattern can have r squared near 1; only a residual plot with no pattern shows a line is the right shape.
  19. r squared is a ratio of two sums of squared pounds, so the units cancel. s is the square root of squared pounds divided by a count, which is pounds.
  20. r squared describes variation, not cause, and not a share of the honey. It says about 76\% of the variation in harvests from hive to hive is accounted for by the line on frames; a strong colony could produce both more comb and more honey.
kg1128 · updated 2026-10-20 · CC-BY-SA-4.0