Reading regression output

When you reach for it

Zsofia measures twelve dragonflies (made-up data), recording body length x and wingspan y, both in millimeters, as pairs (x, y): (42, 65), (49, 77), (50, 72), (51, 70), (52, 82), (53, 80), (55, 83), (57, 75), (62, 93), (64, 94), (66, 95), (71, 103). The dots rise in a fairly tight straight band.

She hands the data to a statistics program, which fits the least-squares line and prints this:

Nobody computes a line through a real data set by hand; a program or calculator does it, and the person reading the result was often not the one who ran it. A printout like this is how a fitted line is reported in articles, lab notes and textbooks. This lesson reads one: where the intercept and slope sit, which entries describe the fit, how to recover r, and which entries belong to later work. Running the fit yourself is a different job, Fitting linear and exponential models with technology; here the output is already in your hands.

The procedure

The line is in the Coef column. Each row of the table is one term of the line \hat{y} = a + bx. The row labeled Constant holds the intercept a, the term with no variable attached. The other row is named after the explanatory variable, and its Coef is the slope b, the number that multiplies that variable. So Zsofia’s line is

A printout names the explanatory variable and leaves the response to be read from the heading or the study, so write the line with both variable names, not with bare x and y. These are the same a and b that the formulas b = r\dfrac{s_y}{s_x} and a = \bar{y} - b\bar{x} produce (Slope from r and the spreads, and regression to the mean); the program has simply done that arithmetic.

S and R-Sq describe the fit. Both were built in r squared and the standard deviation of the residuals. \text{S} is s, the standard deviation of the residuals, \displaystyle s = \sqrt{\text{SSE}/(n - 2)}, in the units of y: the typical size of a prediction error. \text{R-Sq} is r squared, 1 - \text{SSE}/\text{SST}, printed as a percent: the share of the variation in y accounted for by the line.

r comes from R-Sq and the sign of the slope. Most printouts do not show r. Since r squared is the square of r, r is one of \displaystyle \pm\sqrt{r^2}. A square has lost its sign, so the sign comes from elsewhere: r always has the sign of the slope, because b = r\dfrac{s_y}{s_x} and the standard deviations are positive. Convert the percent to a decimal first; 88.5\% is 0.885.

The other entries are for inference, later. The fitted line describes these twelve dragonflies. A natural further question is what the data say about the line for all dragonflies of this kind, and the remaining entries answer that question:

• \text{SE Coef} is the Standard error of each coefficient, a measure of how much that coefficient would vary from sample to sample;
• \text{T} is each coefficient divided by its standard error, for example 8.686 \div 8.494 \approx 1.02 in the Constant row, a test statistic;
• \text{P} is the P-value for that statistic;
• \text{R-Sq(adj)} is r squared adjusted for the number of explanatory variables, which matters when a model has several.

Using any of them needs conditions on how the data were produced and on the residuals (Conditions for regression inference) and the ideas of Test for a slope. Until then, read past them. The Constant row’s \text{P} = 0.331, for example, says nothing about whether the line describes the data well.

Other layouts carry the same numbers. Programs differ in labels: “Estimate” or “Coefficient” for Coef, “(Intercept)” for Constant, “Std. Error” for SE Coef, “Residual standard error” for S, “Multiple R-squared” for R-Sq. A graphing calculator’s linear regression usually prints a short list, a, b, r^2 and r, and no s. Whatever the layout, find the intercept and the slope first, and check which letter is which: some calculators offer the form y = ax + b, where a is the slope and b the intercept, the reverse of \hat{y} = a + bx.

Worked examples

Boundaries

• A printout reports the least-squares line whether or not a line suits the data. It does not check the shape; look at the scatter plot and the residual plot first.
• The line predicts only inside the range of the explanatory variable; the printout does not show that range.
• \displaystyle r = \pm\sqrt{\text{R-Sq}} holds for a line with one explanatory variable. With several, R-Sq is still a share of variation but is not the square of any one correlation.
• \text{SE Coef}, \text{T}, \text{P} and \text{R-Sq(adj)} carry meaning only under the conditions of Conditions for regression inference.
• Labels vary by program. Identify the intercept and the slope from the layout, not from the letters a and b.

Where this goes next

Reading output is the bridge to inference. The slope in the printout belongs to one sample; a different twelve dragonflies would give a different slope, and \text{SE Coef} measures how much. Interval for a slope turns the slope and its standard error into an interval for the true rate, and Test for a slope reads the \text{T} and \text{P} columns. Before trusting any single fit, Outliers, leverage and influential points asks whether one dragonfly is steering the line.

Exercises

Levente counts the dragonflies that pass a pond in 10 minutes on fourteen afternoons with air temperatures from 17 to 32 degrees Celsius (made-up data). Output A:

Zsofia measures ten damselflies, body lengths 29 to 48 mm, response wingspan in mm (made-up data). Output B:

Bence counts dragonflies at the same pond on twelve days with wind speeds from 2 to 18 miles per hour (made-up data). Output C:

Writing and using the line

1. From Output A, write the least-squares line in context.
2. Use Output A to predict the count on a 26-degree afternoon.
3. Interpret the slope of Output A. Does its constant have a meaning in context?
4. From Output B, write the line and predict the wingspan of a damselfly 38 mm long.
5. From Output C, write the line, interpret its slope, and predict the count when the wind is 10 miles per hour.

S and R-Sq

6. Interpret \text{S} and \text{R-Sq} for Output A.
7. One of Zsofia’s damselflies is 41 mm long with a wingspan of 53 mm. Find its residual from Output B and compare it with \text{S}.
8. Of Outputs A, B and C, which line accounts for the largest share of the variation in its response? Interpret \text{R-Sq} for Output C.

Finding r

9. Find r for Output A.
10. Find r for Output C.
11. A printout shows a slope of -0.25 and \text{R-Sq} = 64.0\%. Find r.
12. Printout P shows a slope of 2.1 and \text{R-Sq} = 49.0\%; printout Q shows a slope of -0.6 and \text{R-Sq} = 81.0\%. Find r for each and say which shows the stronger linear association.

Other layouts

13. Bence records the wingbeat rate of eight dragonflies, in beats per second, at air temperatures from 16 to 30 degrees Celsius (made-up data). His calculator shows y = a + bx, a = 21.6549, b = 0.4402, r^2 = 0.8914, r = 0.9442. Write the line in context and give r squared as a percent.
14. For the data in Exercise 13, the wingbeat rates have s_y = 2.3905 beats per second. Find s.
15. A calculator set to the form y = ax + b predicts hindwing length from forewing length, both in mm, and shows a = 1.12 and b = -0.4. Write the line as \hat{y} = a + bx and name the slope and the intercept.
16. A smudged copy of Output A has lost the Constant’s Coef. The temperatures have mean \bar{x} = 24.5 and the counts \bar{y} = 13.643. Recover the constant.
17. List which entries of Output B are used to describe these ten damselflies and which belong to later inference.

Reasoning

18. Explain why \text{R-Sq} alone cannot give the sign of r, and where the sign comes from.
19. Zsofia’s twelve dragonflies are measured again in centimeters instead of millimeters, both variables. Which entries of her printout change, and to what?
20. A classmate reads Zsofia’s \text{R-Sq} = 88.5\% as “the predictions are 88.5\% accurate”. Explain what it says instead.