Transforming to linearity: logs and exponential models
The counts climb faster every year and the least-squares line leaves a curve in its residual plot. Can the ordinary line-fitting machinery still describe data like that?
Where you are 9 in Bivariate data
Why this exists
The least-squares line is the model regression hands you ready-made: formulas for a and b, residuals that balance about 0, and r squared and s to summarize the fit. Much data does not lie along a line. A population that grows by a fixed percent each year adds more animals every year than the year before, so its scatter plot bends upward and the residual plot of the straight-line fit shows a U.
One could invent a new fitting method for every curve. Re-expression is the cheaper idea: change the scale of a variable so the curve becomes straight, fit the ordinary least-squares line on the new scale, judge it with the usual residual plot, then translate the line back into a curve on the original scale. For growth by a constant percent, the change of scale is the logarithm of the response. This lesson follows one made-up data set, manatee counts at a warm-water spring, through all of it: on Day 1 the transformation and the fit, on Day 2 judging the fit and predicting with it. Power models, which need the logarithm of both variables, come next in Log-log transforms and power models.
Definition
To transform (re-express) a variable is to replace every value y by f(y) for one fixed function f, keeping each case paired with its own x. The log transformation of the response replaces y by \log y (base 10) or \ln y (base e), and needs every y to be positive. Recall from Logarithms that \log y is the power 10 must be raised to in order to give y, so \log 1000 = 3.
Fitting an exponential model from data means computing the least-squares line of \log y on x,
and then undoing the logarithm, which turns the line into a curve:
Precalculus writes an exponential model as y = ab^x. In regression, a and b already name the intercept and slope of a least-squares line, so the exponential’s constants are written A and B here. A is the predicted y at x = 0, and B is the factor the prediction is multiplied by for each one-unit step in x. With natural logs, A = e^{a} and B = e^{b}.
Intuition
Growth by about 6\% a year means each count is about 1.06 times the one before. Logarithms turn multiplying into adding: \log(1.06\,y) = \log 1.06 + \log y \approx 0.0253 + \log y. So on the log scale every year adds about the same amount, and adding a constant amount per step is exactly what a straight line does. Semi-log plots: straightening exponential data with logarithms shows this picture for exponential data with almost no scatter. With real scatter, the least-squares line on (x, \log y) is the best line on that scale, and the residual plot does the judging twice: a curved residual plot of the raw fit is the reason to transform, and a residual plot with no pattern for the transformed fit is the evidence it worked.
Results
Exponential in, straight line out. If y = A B^x exactly, with A > 0 and B > 0, then
a straight line in x with intercept \log A and slope \log B.
Back-transforming. The line \widehat{\log y} = a + bx becomes \hat{y} = 10^{a}\,(10^{b})^x. Each one-unit increase in x adds b to the predicted \log y and multiplies the predicted y by 10^{b}, a change of (10^{b} - 1) \times 100 percent.
The base does not matter. Fitting \ln y instead of \log y multiplies the intercept and the slope by \ln 10 \approx 2.3026, and gives the same A, the same B, the same predictions and the same r squared.
What r squared measures. The r squared of the line on (x, \log y) is the fraction of the variation in \log y that the line accounts for. It is not a statement about y, so it cannot be set beside the r squared of the raw line as if both measured the same thing.
Why it is true
Line out. By the product and power rules for logarithms, \log(A B^x) = \log A + \log(B^x) = \log A + x \log B. Here \log A and \log B are fixed numbers, so this is a line in x.
Back-transforming. Since 10^{\log y} = y, raising 10 to each side of the fitted line gives the prediction on the original scale. The exponent laws 10^{m + n} = 10^{m}\,10^{n} and 10^{mn} = (10^{m})^{n} give 10^{a + bx} = 10^{a}\,(10^{b})^x. Moving from x to x + 1 multiplies this by 10^{b} once more.
Base. Every positive y is 10^{\log y}, so \ln y = (\log y)(\ln 10): each natural log is the common log times the constant k = \ln 10. Multiplying every response by k multiplies s_y and \bar{y} by k and leaves r unchanged, so the slope b = r\,s_y / s_x and the intercept a = \bar{y} - b\bar{x} are both multiplied by k. Then e^{ka} = (e^{\ln 10})^{a} = 10^{a}, and in the same way e^{kb} = 10^{b}. Since r is unchanged, so is r squared.
r squared. r squared is computed from the pairs it is given. Given (x, \log y), it compares the squared residuals of \log y with the spread of \log y about its own mean, and y never enters.
Worked examples
Day 1: the straight line leaves a curve
Hiwot, Yonas and Mekdes count the manatees at a warm-water spring every winter (made-up data). As pairs (years since the first count x, manatees counted y): (0, 1212), (1, 1259), (2, 1356), (3, 1421), (4, 1527), (5, 1593), (6, 1702), (7, 1817), (8, 1899), (9, 2038), (10, 2130), (11, 2296), (12, 2400). Technology gives the least-squares line \hat{y} = 1141.4 + 100.2x with r = 0.995. Is a line a fair model?
SolutionThe correlation is high, but r measures how tightly the dots hug a line, not whether a line is the right shape, so make the residual plot. Each residual is actual minus predicted; for x = 0 the line predicts 1141.4 and the residual is 1212 - 1141.4 = 70.6.
| x | y | \hat{y} | y - \hat{y} |
|---|---|---|---|
| 0 | 1212 | 1141.4 | 70.6 |
| 1 | 1259 | 1241.6 | 17.4 |
| 2 | 1356 | 1341.8 | 14.2 |
| 3 | 1421 | 1442.0 | -21.0 |
| 4 | 1527 | 1542.2 | -15.2 |
| 5 | 1593 | 1642.4 | -49.4 |
| 6 | 1702 | 1742.6 | -40.6 |
| 7 | 1817 | 1842.8 | -25.8 |
| 8 | 1899 | 1943.0 | -44.0 |
| 9 | 2038 | 2043.2 | -5.2 |
| 10 | 2130 | 2143.4 | -13.4 |
| 11 | 2296 | 2243.6 | 52.4 |
| 12 | 2400 | 2343.8 | 56.2 |
The residuals are positive for the first three years, negative for every year from 3 to 10, and positive again at the end: a U. The counts bend upward away from the line, which cuts across the bend. The data say why. Over the first six years the count rose by \displaystyle \frac{1702 - 1212}{6} \approx 81.7 manatees a year on average, over the last six by \displaystyle \frac{2400 - 1702}{6} \approx 116.3. The gains grow as the herd grows, which is the mark of growth by a percent rather than by a fixed number. A line is the wrong shape, whatever r says.
Day 1: taking logs and fitting the line
Take the base-10 log of every count, fit the least-squares line of \log y on x, and say what its slope means.
SolutionEach log is computed to four decimal places; for example \log 1212 \approx 3.0835, because 10^{3.0835} \approx 1212. The year stays as it was.
| x | y | \log y |
|---|---|---|
| 0 | 1212 | 3.0835 |
| 1 | 1259 | 3.1000 |
| 2 | 1356 | 3.1323 |
| 3 | 1421 | 3.1526 |
| 4 | 1527 | 3.1838 |
| 5 | 1593 | 3.2022 |
| 6 | 1702 | 3.2310 |
| 7 | 1817 | 3.2594 |
| 8 | 1899 | 3.2785 |
| 9 | 2038 | 3.3092 |
| 10 | 2130 | 3.3284 |
| 11 | 2296 | 3.3610 |
| 12 | 2400 | 3.3802 |
Technology, given the pairs (x, \log y), returns
The dots now lie in a straight band. The slope says that each additional year raises the predicted log of the count by 0.0252. The intercept, 3.0797, is the predicted log of the count in the year of the first count. Both are in log units: the line describes \log y, and the next example turns it back into manatees.
Day 1: back to a curve
Turn the line of Example 2 into a model \hat{y} = A B^x and interpret A and B. Mekdes fitted the same data with natural logs and got \widehat{\ln y} = 7.0914 + 0.0580x. Show that her model is the same.
SolutionUndo the log by raising 10 to each side: A = 10^{3.0797} \approx 1201.4 and B = 10^{0.0252} \approx 1.0597, so
The model predicts about 1201 manatees in the year of the first count, and multiplies its prediction by 1.0597 each year, a growth of about 6.0\% a year. Notice A is not the first count, 1212: the line passes near every point, not through the first one.
For Mekdes’s line, undo the natural log with e: e^{7.0914} \approx 1201.6 and e^{0.0580} \approx 1.0597. Her coefficients are those of Example 2 times \ln 10 (3.0797 \times 2.3026 \approx 7.0913 and 0.0252 \times 2.3026 \approx 0.0580), up to the rounding of the printed coefficients, and the two models agree to that same rounding.
Day 2: judging the transformed fit
Make the residual plot of the line \widehat{\log y} = 3.0797 + 0.0252x and judge it. What does a residual on this scale mean in manatees?
SolutionThe residual is \log y - \widehat{\log y}. For x = 1: 3.1000 - (3.0797 + 0.0252) = -0.0049.
| x | \log y | \widehat{\log y} | residual |
|---|---|---|---|
| 0 | 3.0835 | 3.0797 | 0.0038 |
| 1 | 3.1000 | 3.1049 | -0.0049 |
| 2 | 3.1323 | 3.1301 | 0.0022 |
| 3 | 3.1526 | 3.1553 | -0.0027 |
| 4 | 3.1838 | 3.1805 | 0.0033 |
| 5 | 3.2022 | 3.2057 | -0.0035 |
| 6 | 3.2310 | 3.2309 | 0.0001 |
| 7 | 3.2594 | 3.2561 | 0.0033 |
| 8 | 3.2785 | 3.2813 | -0.0028 |
| 9 | 3.3092 | 3.3065 | 0.0027 |
| 10 | 3.3284 | 3.3317 | -0.0033 |
| 11 | 3.3610 | 3.3569 | 0.0041 |
| 12 | 3.3802 | 3.3821 | -0.0019 |
Positive and negative residuals are mixed along the whole range, with no U, no arch and no widening. The U of Example 1 is gone, so the log transformation has straightened the data and the line on the log scale is a fair model.
A residual in log units is a ratio in manatees. Since \log y - \widehat{\log y} = \log(y / \hat{y}), a residual of 0.0041 (year 11) says the count was 10^{0.0041} \approx 1.0095 times the prediction, about 0.9\% above it. The most negative, -0.0049, gives 10^{-0.0049} \approx 0.9888, about 1.1\% below. Every count is within about 1.1\% of the model.
Day 2: comparing the two fits
The straight line has r squared 0.990; the line on the log scale has r squared 0.999. Does that comparison settle which model is better? Compare the two models on the scale of the counts.
SolutionNo. The first r squared says the line accounts for 99.0\% of the variation in the counts; the second says the log line accounts for 99.9\% of the variation in the log counts. They describe different responses. What settles the choice is the pair of residual plots: a U for the line, no pattern for the exponential.
To put both models on one scale, compute the exponential model’s residuals in manatees, y - \hat{y} with \hat{y} = 10^{3.0797 + 0.0252x}. For year 0, \hat{y} = 10^{3.0797} \approx 1201.4, so the residual is 1212 - 1201.4 = 10.6. All thirteen, to one decimal place:
Their standard deviation, found the same way as s for a line (square, add, divide by n - 2 = 11, take the square root), is about 13.7 manatees. The straight line’s s is about 41.5 manatees. The exponential model’s predictions are typically about three times closer, and its misses in manatees also show no pattern. These count residuals add to about 2.0, not 0: the least-squares line balanced the log residuals, not these.
Day 2: predicting with the model
Predict the count in year 13, the next winter, and in year 20, with both models. Which predictions deserve trust?
SolutionPredict on the log scale, then back-transform; this avoids the rounding in A and B. For year 13, \widehat{\log y} = 3.0797 + 0.0252(13) = 3.4073, so \hat{y} = 10^{3.4073} \approx 2554 manatees. The line gives 1141.4 + 100.2(13) = 2444.0. For year 20, \widehat{\log y} = 3.5837 and \hat{y} \approx 3834; the line gives 3145.4.
At year 13 the exponential is the better guess: the line’s residuals were positive and rising at the right end of its U, so it was already falling behind the counts, while the exponential left no pattern. Year 13 is still one step beyond the data, so even this prediction assumes the growth continues. Year 20 is eight years past the last count, an extrapolation. The two models already disagree by nearly 700 manatees there, and nothing in the data shows whether the spring can hold that many: warm water and food are limited, and growth by a constant percent cannot go on forever.
Boundaries
- The log transformation needs every y > 0. A count of 0 or a negative value has no logarithm.
- It straightens growth or decay by a roughly constant factor per unit of x, and nothing else. A curve in the residual plot of the transformed fit says the log was the wrong re-expression (Exercise 12). Power relationships need logs of both variables: Log-log transforms and power models.
- The least-squares line on the log scale makes the squared log residuals small, not the squared residuals in y. Its residuals in y need not add to 0 (Example 5).
- Back-transforming an average log does not give an average y: 10^{\widehat{\log y}} estimates the geometric mean of y at that x (also the median, when the log values scatter symmetrically about the line), and that sits below the mean of y there whenever the values scatter at all. With scatter as small as the manatee counts’ the difference is negligible.
- r squared on \log y is about \log y. Compare models by their residual plots, or by residuals converted to the same scale.
- Predictions beyond the data assume the pattern continues, and exponential predictions grow fastest exactly there.
Where this goes next
The same plan, transform, fit, judge, back-transform, works with logs of both variables, which straightens power models such as y = A x^{p} in Log-log transforms and power models. The numbers used to compare fits, r squared and s, are studied in their own right in r squared and the standard deviation of the residuals, and the model A B^x itself is the subject of Exponential growth and decay. Later, inference for regression reads the same residual plots on whatever scale the line was fitted, so a transformation chosen here carries into the conditions checked in Conditions for regression inference.
Exercises
Day 1: logs and the fitted line
- In two winters a spring had 1450 and then 1537 manatees. Find the log of each count to four decimal places and their difference. What growth factor does the difference stand for?
- For a smaller spring Yonas fits \widehat{\log y} = 2.6 + 0.04x, with x in years. Write the model as \hat{y} = A B^x and interpret A and B.
- The area of seagrass in a lagoon, in hectares, is fitted with \widehat{\ln y} = 6.9 - 0.11x, with x in years. Write the model as \hat{y} = A B^x and give the percent change per year.
- Mekdes measures the seagrass where the manatees feed each year (made-up data), as pairs (years, hectares): (0, 1480), (1, 1375), (2, 1240), (3, 1159), (4, 1054), (5, 983), (6, 890). Use technology to fit the least-squares line of \log y on x, with coefficients to four decimal places, and write the model as \hat{y} = A B^x.
- A line on the log scale has intercept 3.2. What does the back-transformed model predict at x = 0?
- Yonas fits \widehat{\log y} = 3.05 + 0.03x. What intercept and slope will Mekdes get fitting \ln y to the same data? Show the two models are the same.
- The slope of a line on the log scale is b = 0.0414 per year. By what percent does the model’s prediction change each year?
Day 2: judging and predicting
- On the log scale, one residual is 0.02 and another is -0.01. Say what each means as a percent of the prediction.
- Using Yonas’s model from Exercise 2, predict the count at x = 5. The actual count was 650. Find the residual in manatees and in log units.
- Using the line from Exercise 4, predict the seagrass area in year 8. The least-squares line on the raw areas is \hat{y} = 1462.3 - 97.9x. What does it predict in year 8, and in what year does it predict no seagrass at all? Which model is easier to believe?
- For the seagrass data, the line on the area has r squared 0.993 and the line on the log area has r squared 0.998. Hiwot says that proves the log model is better. Respond.
- The photo catalog of manatees identified at the spring grows each year (made-up data), as pairs (years, manatees in the catalog): (0, 210), (1, 252), (2, 289), (3, 331), (4, 368), (5, 412), (6, 449), (7, 490). Mekdes fits \log y on x; the residuals are -0.0255, 0.0021, 0.0099, 0.0172, 0.0116, 0.0090, -0.0053, -0.0190. The line \hat{y} = 210.5 + 39.9x has residuals -0.5, 1.6, -1.3, 0.8, -2.1, 2.0, -0.9, 0.2. Describe each residual plot and choose a model.
- Use the manatee line \widehat{\log y} = 3.0797 + 0.0252x to predict the count in year 30. Give two reasons to distrust the answer.
Reasoning
- Explain why growth by the same percent each year gives a straight line on the log scale, while growth by the same number each year does not.
- Explain why fitting \ln y and fitting \log y give the same predictions.
- At a tiny spring one winter’s count was 0. Why can the log transformation not be used on those counts as they stand?
- A line’s residual plot is an arch because the response rises quickly and then levels off. Would taking \log y straighten the data? Explain.
- In Example 5 the exponential model’s residuals in manatees add to about 2.0. Why is that not a mistake?
- Why is the back-transformed A in Example 3 not equal to the first count?
Answers
- \log 1450 \approx 3.1614 and \log 1537 \approx 3.1867; the difference is 0.0253. That is \log 1.06, since \displaystyle \frac{1537}{1450} = 1.06: a growth factor of 1.06, a 6\% rise.
- A = 10^{2.6} \approx 398.1 and B = 10^{0.04} \approx 1.0965, so \hat{y} = 398.1(1.0965)^x. The model predicts about 398 manatees at x = 0 and a rise of about 9.6\% a year.
- A = e^{6.9} \approx 992.3 and B = e^{-0.11} \approx 0.8958, so \hat{y} = 992.3(0.8958)^x. Each year the predicted area is multiplied by 0.8958, a loss of about 10.4\% a year.
- \widehat{\log y} = 3.1713 - 0.0366x. Then A = 10^{3.1713} \approx 1483.5 and B = 10^{-0.0366} \approx 0.9192: \hat{y} = 1483.5(0.9192)^x, a loss of about 8.1\% a year.
- 10^{3.2} \approx 1585.
- Multiply both by \ln 10 \approx 2.3026: \widehat{\ln y} = 7.0229 + 0.0691x. Then e^{7.0229} \approx 1122.0 \approx 10^{3.05} and e^{0.0691} \approx 1.0715 \approx 10^{0.03}, the same A and B.
- 10^{0.0414} \approx 1.1000, so the prediction rises by about 10\% a year.
- 10^{0.02} \approx 1.047: the actual value is about 4.7\% above the prediction. 10^{-0.01} \approx 0.977: about 2.3\% below.
- \widehat{\log y} = 2.6 + 0.04(5) = 2.8, so \hat{y} = 10^{2.8} \approx 631. The residual is 650 - 631 = 19 manatees, and on the log scale \log 650 - 2.8 \approx 0.0129.
- \widehat{\log y} = 3.1713 - 0.0366(8) = 2.8785, so \hat{y} \approx 756 hectares. The line gives 1462.3 - 97.9(8) = 679.1 hectares and reaches 0 at \displaystyle x = \frac{1462.3}{97.9} \approx 14.9 years. The exponential is easier to believe: it shrinks by a fixed percent and never goes below 0, while the line soon predicts a negative area. Both are extrapolations.
- The two r squared values describe different responses, area and log of area, so a larger one does not by itself make a better model. Compare the residual plots of the two fits, or convert the log model’s residuals to hectares and compare them with the line’s.
- The log fit’s residuals are negative at both ends and positive in the middle: an arch, so the log bent the data too far. The line’s residuals are small, mixed in sign, with no shape. The catalog grows by about 40 manatees a year, a fixed number, so the line is the model to use.
- \widehat{\log y} = 3.0797 + 0.0252(30) = 3.8357, so \hat{y} \approx 6850 manatees. Year 30 is 18 years past the data, and a spring cannot hold an ever-growing herd: growth by a fixed percent must slow at some point.
- A fixed percent multiplies y by the same factor B each year, and \log(By) = \log B + \log y, so \log y goes up by the same amount, \log B, each year: a line. Adding a fixed number d gives \displaystyle \log(y + d) - \log y = \log\left(1 + \frac{d}{y}\right), which shrinks as y grows, so the log curve flattens.
- Every natural log is the common log times \ln 10, so the fitted intercept and slope are both multiplied by \ln 10, and e^{(\ln 10)a} = 10^{a}. The back-transformed model, and so every prediction, is the same.
- \log 0 is not defined: no power of 10 equals 0.
- No. Taking logs pulls large values down more than small ones, so a curve that is already leveling off levels off even more. The arch would remain, deeper. Leveling off calls for a different model.
- Least squares on the log scale makes the log residuals balance. The model on the count scale is a curve, not the least-squares line for the counts, so its residuals there need not add to 0.
- A = 10^{a}, and a is the least-squares intercept, set by all thirteen points, not by the first one. The first count has its own residual, 0.0038 on the log scale, so it sits above the model.