Fitting linear and exponential models with technology
Two people draw two different lines through the same scatter plot. Is there one line, or one curve, that everybody can agree on, and how does a calculator find it?
Where you are 5 in Least squares regression
When you reach for it
A Line of best fit drawn by eye is quick, but two careful people draw two slightly different lines, and neither can say why theirs is the right one. When you have paired data on a scatter plot (see Plotting paired data: explanatory and response variables) with a clear trend (Describing a scatter plot), a calculator or spreadsheet can fit one model that anyone using the same numbers will get, in seconds.
On Day 1 that model is a straight line, the least-squares line. On Day 2 it is an exponential curve y = ab^x, for data that grow or shrink by roughly constant ratios. The skill is not pressing the buttons. It is knowing what the machine chose, reading its numbers in context, and deciding which of the two models the data and the situation call for. Every data set in this lesson is made up.
The procedure
Day 1: the least-squares line
What the machine makes small. Take any line through a scatter plot. Each data point has a vertical gap to it: the point’s actual y minus the line’s y at the same x. A point above the line has a positive gap, a point below a negative one.
Adding the gaps does not judge a line, because positive and negative gaps cancel: every line through the center of the cloud, the point of means (\bar{x}, \bar{y}) below, has gaps that total zero, however badly it is tilted. So each gap is squared, which makes it positive and makes a big miss count far more than a small one, and the squares are added. The least-squares line is the line whose total of squared vertical gaps is as small as possible. For any data with at least two different x-values there is exactly one such line, and technology finds it with formulas you do not need here. Later this unit the gaps get a name, residuals, in Interpreting a residual.
Entering the data. Type the data as two lists of the same length, one pair per row: the x-values (the explanatory variable, the one you predict from) in the first list and the y-values in the second. The order matters. The gaps are measured in the y direction, so swapping the lists measures them sideways instead and gives a different line, not the same line rearranged. Then choose linear regression (a spreadsheet calls it a linear trend line).
Reading the output. Many calculators show the form first and the numbers under it, like this for the queue data of Example 1, rounded here to four decimal places:
In this form a is the slope and b the intercept. Some machines use y = a + bx, where the letters swap jobs, so always read the form at the top. The screen also shows a number r; it has its own lesson, The correlation coefficient, and nothing here uses it.
Writing the equation in context. Put the numbers into the form, round sensibly (three significant figures is usually plenty), and say what x and y stand for, with units. The left side is written \hat{y}, read “y-hat”, which means the predicted value of y. The hat is a reminder that the line gives a prediction, not the measured value: the dots are the data, the line is the model.
A check you can always make. The least-squares line always passes through the point of means, (\bar{x}, \bar{y}), where \bar{x} is the mean of the x-values and \bar{y} the mean of the y-values. Putting \bar{x} into your equation should give \bar{y}, up to rounding. If it does not, a number was copied wrongly.
Day 2: exponential fits
When a scatter plot bends, and equal steps in x multiply y by roughly the same ratio (the test in Choosing a function family for data), a straight line is the wrong family. Enter the same two lists and choose exponential regression instead. The output looks like
and the letters now mean something different from Day 1, as reading a model’s parameters explains:
- a is the start value, the model’s value of y when x = 0, since b^0 = 1.
- b is the factor for each step of 1 in x. If b > 1 the quantity grows by (b - 1) \times 100\% per step; if 0 < b < 1 it decays, losing (1 - b) \times 100\% per step.
Finding y = ab^x by hand uses two points and ignores the rest. Technology uses all of them: it also keeps the misses small, measured in a way suited to growth by ratios, and how it does that is beyond this lesson. So its b is usually close to the ratios in the data without being any one of them.
Choosing between a line and an exponential. Technology fits whichever model you ask for, to any data, and never warns you that you asked for the wrong one. The choice is yours, made from two things, not from a number on the screen:
- The plot. Does the cloud lie along a straight band or bend? Fit a line to bending data and the dots sit on one side of the line at both ends and on the other side in the middle. Making and reading residual plots makes that pattern sharper.
- The context. Does the quantity change by about the same amount each step (a steady rate), or by about the same fraction of itself (each member brings in more members, the body clears a share of what is there)? And does each model tell a possible story where it should, such as a start value that could really happen?
Worked examples
Day 1: fitting a line to queue data
Candela noted the number of people in the queue for a ride and the wait posted at the entrance, on eight visits (made-up data).
| people in the queue, x | posted wait (minutes), y |
|---|---|
| 20 | 6 |
| 35 | 10 |
| 50 | 14 |
| 60 | 15 |
| 80 | 22 |
| 95 | 25 |
| 120 | 32 |
| 140 | 36 |
Use technology to fit the least-squares line, and write its equation in context.
SolutionPeople go in the first list because the wait is what we predict from the queue. Linear regression gives the output shown above: a = 0.2539 and b = 0.9568 in the form y = ax + b, so a is the slope and b the intercept. Rounding the slope to three significant figures and the intercept to two decimal places,
where x is the number of people in the queue and \hat{y} is the predicted posted wait in minutes.
The dots lie close to a straight band, so a line is a sensible model here.
Day 1: what the queue line says
Using \hat{y} = 0.254x + 0.96 from Example 1: (a) say what the slope and intercept mean; (b) check that the line passes through (\bar{x}, \bar{y}); (c) predict the posted wait when 70 people are queuing.
Solution(a) Reading a fitted slope and intercept is the subject of Interpreting the slope and intercept; here it is in brief. The slope says each extra person in the queue adds about 0.254 minutes, roughly 15 seconds, to the predicted wait, so about four more people mean about one more minute. The intercept says an empty queue predicts a wait of about 0.96 minutes. No visit had fewer than 20 people, so that is a reach beyond the data, though a posted wait of about a minute for nobody waiting (the ride itself) is believable.
(b) The x-values total 600, so \displaystyle \bar{x} = \frac{600}{8} = 75. The y-values total 160, so \displaystyle \bar{y} = \frac{160}{8} = 20. Then
which is \bar{y} = 20 up to the rounding of the slope. The ring in the figure sits on the line.
(c) 70 lies inside the data, between 60 and 80, so the line can be trusted there: \hat{y} = 0.254(70) + 0.96 = 18.74, a posted wait of about 18.7 minutes. How far such predictions can be pushed is Predicting with a regression line, and extrapolation.
Day 1: least squares against a line by eye
A bus route has these journey times (made-up data).
| stops, x | journey time (minutes), y |
|---|---|
| 2 | 9 |
| 4 | 12 |
| 6 | 18 |
| 8 | 20 |
| 10 | 26 |
Technology gives \hat{y} = 2.1x + 4.4. A line drawn by eye through the first and last dots is \hat{y} = 2.125x + 4.75. Find the total of squared vertical gaps for each line.
SolutionWork out each line’s \hat{y} at every x (“fit” is the least-squares line), subtract it from the actual y, and square.
| x | y | fit \hat{y} | gap | by eye \hat{y} | gap |
|---|---|---|---|---|---|
| 2 | 9 | 8.6 | 0.4 | 9 | 0 |
| 4 | 12 | 12.8 | -0.8 | 13.25 | -1.25 |
| 6 | 18 | 17 | 1 | 17.5 | 0.5 |
| 8 | 20 | 21.2 | -1.2 | 21.75 | -1.75 |
| 10 | 26 | 25.4 | 0.6 | 26 | 0 |
Least squares: 0.16 + 0.64 + 1 + 1.44 + 0.36 = 3.6. By eye: 0 + 1.5625 + 0.25 + 3.0625 + 0 = 4.875.
The by-eye line passes exactly through two dots, yet its total is larger: hitting some points exactly does not make a line fit the whole cloud best. Notice also that the least-squares gaps total 0.4 - 0.8 + 1 - 1.2 + 0.6 = 0, which is why the gaps have to be squared before they can compare lines. The two lines are close; a good line by eye is a fair guess at the least-squares line, and technology removes the guessing.
Day 2: caffeine leaving the body
Caspian drank a strong coffee. Estimates of the caffeine left in his body, hour by hour, are below (made-up figures).
| hours, x | caffeine (mg), y |
|---|---|
| 1 | 140 |
| 2 | 122 |
| 3 | 105 |
| 4 | 93 |
| 5 | 80 |
| 6 | 70 |
| 7 | 61 |
Fit an exponential model and say what a and b mean. About how long does it take for half the caffeine to go?
SolutionFirst, is exponential the right family? The ratios of neighboring values are 0.871, 0.861, 0.886, 0.860, 0.875 and 0.871, all close to one another, and the plot bends, dropping by less each hour. Exponential regression gives a = 160.5195 and b = 0.8707, so
where x is hours since the coffee and \hat{y} the predicted caffeine in milligrams.
a = 160.5: the model’s amount at hour 0, about 160 mg, the caffeine in the coffee. There was no measurement at hour 0; this is what the trend points back to.
b = 0.871: each hour about 87.1\% of the caffeine is still there, so about 12.9\% of what is left goes each hour.
Half: the factor over 5 hours is 0.871^5 \approx 0.501, so about half goes every 5 hours. The data agree: 70 mg at hour 6 is half the 140 mg at hour 1.
Day 2: a club growing week by week
Brigitta started an online club and counted members every two weeks (made-up data).
| week, x | members, y |
|---|---|
| 0 | 30 |
| 2 | 52 |
| 4 | 86 |
| 6 | 150 |
| 8 | 250 |
| 10 | 420 |
Exponential regression gives a = 30.2993 and b = 1.3018. Write the model, say what b means, and estimate the membership at week 5.
Solutionwith x in weeks and \hat{y} the predicted number of members. The start value 30.3 is very close to the 30 members counted at week 0.
b is the factor for one step of 1 in x, and x counts weeks, not readings. So the club grows by a factor of about 1.302, that is about 30\%, each week. Over one gap between readings, two weeks, the factor is 1.302^2 \approx 1.695, which matches the ratios in the table: \displaystyle \frac{52}{30} \approx 1.73, \displaystyle \frac{86}{52} \approx 1.65, \displaystyle \frac{150}{86} \approx 1.74, \displaystyle \frac{250}{150} \approx 1.67, \displaystyle \frac{420}{250} = 1.68.
Week 5 lies inside the data, so
about 113 members.
Day 2: a line or a curve for the club?
Brigitta also asks for linear regression on the club data and gets a = 37.2571, b = -21.6190. Which model should she use?
SolutionThe line is \hat{y} = 37.26x - 21.62. Technology produced it without complaint, which says nothing about whether it fits. Judge it by the plot and the context.
The plot: the line’s values at weeks 0, 2, 4, 6, 8, 10 are about -21.6, 52.9, 127.4, 201.9, 276.4 and 351.0. The dots sit above the line at both ends and below it in the middle, the pattern of a line forced onto a bend. The exponential curve has no such pattern.
The context: the line predicts -21.6 members at week 0, which cannot happen. It also says the club gains the same 37 members every week, while a club whose members bring in friends grows by a share of its size, and the near-constant ratios say exactly that. Use the exponential model.
Day 2: a line or a curve for a fuel tank?
Conrad drives a delivery van along a highway at a steady speed and notes the fuel gauge every hour (made-up data).
| hours, x | fuel (% of full), y |
|---|---|
| 0 | 96 |
| 1 | 84.4 |
| 2 | 72 |
| 3 | 60.8 |
| 4 | 48.4 |
| 5 | 36 |
Linear regression gives a = -11.9771, b = 96.2095. Exponential regression gives a = 101.7714, b = 0.8248. Which model fits the situation?
SolutionThe two models are \hat{y} = -11.98x + 96.21 and \hat{y} = 101.8(0.825)^x.
The differences are -11.6, -12.4, -11.2, -12.4, -12.4, all close to -12, while the ratios drift from 0.88 down to 0.74. The plot is a straight band falling at a steady rate. The context agrees: at a steady speed the engine burns about the same amount of fuel each hour.
The exponential model also tells an impossible story: its start value is 101.8\%, more than full, and it misses the reading at hour 0 by almost 6 points where the line misses by 0.2. Use the line: the tank loses about 12 percentage points of fuel an hour.
Boundaries
A fit always exists, so it proves nothing. The least-squares line exists for any data with two different x-values, curved or scattered, and an exponential fit exists for any positive data. A fitted model describes the data only when the plot and the context agree with its family.
Gaps are vertical. The line minimizes misses in y. With the lists swapped the result is a different line, so the variable you predict goes second.
Squaring gives one far-off point a large pull. A single point well away from the rest can tilt the least-squares line noticeably; look at the plot before trusting the numbers.
An exponential fit needs every y positive. A calculator refuses a 0 or a negative value, since no ab^x with a > 0 ever reaches them.
a and b depend on how x is counted. b is the factor per step of 1 in x, and a is the value at x = 0. Enter years as 2020, 2021, and a becomes a value for the year 0; count years from the start instead.
The model is only as good as its range. Inside the data it predicts reasonably; outside, the trend may stop (Where a model holds: domain, interpolation and extrapolation).
Where this goes next
The vertical gaps become residuals in Interpreting a residual, and plotting them in Making and reading residual plots turns “the dots sit above at both ends” into a picture you can read at a glance. The r on the screen is explained in The correlation coefficient, and Slope from r and the spreads, and regression to the mean shows how the slope comes from r and the spreads. Using the fitted line to predict, and knowing when not to, is Predicting with a regression line, and extrapolation. When there is no data at all, only a description, Building a model from a description builds the model from the words instead.
Exercises
Day 1: reading the output
Cesarina weighed bags of apples (made-up data).
| apples, x | mass (kg), y |
|---|---|
| 4 | 0.72 |
| 6 | 1.10 |
| 7 | 1.22 |
| 9 | 1.65 |
| 10 | 1.78 |
| 12 | 2.15 |
Linear regression in the form y = ax + b gives a = 0.1788, b = 0.0062.
- Write the equation of the least-squares line for the apples in context.
- Say what the slope of the apples line means.
- Say what the intercept of the apples line means, and whether to trust it.
- Check that the apples line passes through (\bar{x}, \bar{y}).
Day 1: fitting and comparing
- A soup stall recorded the outdoor temperature and the bowls of hot soup sold (made-up data): temperature in °C 2, 5, 8, 11, 14, 17, 20 and bowls 88, 80, 71, 66, 54, 49, 40. Use technology to fit the least-squares line, write it in context, and say what the slope means.
- For the data (1, 3), (2, 4), (3, 8), (4, 9), find the total of squared vertical gaps for line A, \hat{y} = 2.2x + 0.5, and for line B, \hat{y} = 2x + 1. Which could be the least-squares line?
- A data set has \bar{x} = 8 and \bar{y} = 1.4. Could \hat{y} = 0.15x + 0.1 be its least-squares line? Could \hat{y} = 0.2x - 0.2?
- Suppose the Example 1 queue data are entered with the waits in the first list and the people in the second. Explain why the result is not \hat{y} = 0.254x + 0.96, even rearranged.
Day 2: exponential fits
Lemonade is poured and the height of its foam is measured each minute (made-up data).
| minutes, x | foam (mm), y |
|---|---|
| 1 | 40 |
| 2 | 35 |
| 3 | 31 |
| 4 | 27 |
| 5 | 24 |
| 6 | 21 |
Exponential regression gives a = 45.3898, b = 0.8796.
- Write the foam model in context.
- Say what a and b mean for the foam.
A short video’s total views were counted every three days (made-up data).
| day, x | views, y |
|---|---|
| 0 | 200 |
| 3 | 340 |
| 6 | 580 |
| 9 | 950 |
| 12 | 1600 |
- Use technology to fit an exponential model to the views.
- By what percent do the views grow each day, by the model? What factor does it give for three days, and does that match the table?
Day 2: choosing a model
Say whether a line or an exponential fits each data set better, using the numbers and the context.
- Kilometers left on a train journey, every 10 minutes: 30, 26.1, 22.0, 18.2, 14.0.
- A bike’s resale value in dollars, year by year: 400, 340, 290, 245, 210.
- A bus pass costs 8.00, 8.40, 8.80, 9.30 and 9.70 dollars in five successive years. Can the data decide between a line and an exponential? What might?
Reasoning
- Explain why the least-squares line adds the squares of the gaps rather than the gaps themselves.
- A friend says: “My calculator gave me an exponential fit, so the data must be exponential.” What is wrong with this?
- In Example 4 the fitted b, 0.871, equals only some of the ratios in the table. Explain why a fitted b need not equal any ratio in the data.
- In Example 7 the exponential fit’s a is 101.8, though the reading at hour 0 was 96. Explain why a fitted start value can differ from the first data value.
Answers
- \hat{y} = 0.179x + 0.006, where x is the number of apples and \hat{y} the predicted mass in kilograms.
- Each extra apple adds about 0.179 kg, about 179 g, to the predicted mass.
- With no apples the line predicts 0.006 kg, about 6 g. Zero apples is outside the data (4 to 12), so this is a reach, but a number so near 0 is believable: no apples, almost no mass.
- \displaystyle \bar{x} = \frac{48}{6} = 8 and \displaystyle \bar{y} = \frac{8.62}{6} \approx 1.437. The line gives 0.1788(8) + 0.0062 = 1.4366, which matches.
- \hat{y} = -2.65x + 93.2 (technology: a = -2.6548, b = 93.2024), with x the temperature in °C and \hat{y} the predicted bowls sold. Each degree warmer predicts about 2.65 fewer bowls. Check: \bar{x} = 11, \bar{y} = 64, and -2.6548(11) + 93.2024 \approx 64.0.
- A: gaps 0.3, -0.9, 0.9, -0.3, total of squares 1.8. B: gaps 0, -1, 1, 0, total 2. Only A could be the least-squares line, since B’s total is larger.
- The first, no: 0.15(8) + 0.1 = 1.3, not 1.4, and the least-squares line must pass through (8, 1.4). The second could be: 0.2(8) - 0.2 = 1.4. Passing through the point of means is needed but does not prove it.
- With the lists swapped the calculator makes the squared gaps in the people direction as small as possible, measured sideways on the original plot. That is a different requirement, so it picks a different line, not \hat{y} = 0.254x + 0.96 solved for x.
- \hat{y} = 45.4(0.880)^x, where x is minutes since pouring and \hat{y} the predicted foam height in millimeters.
- a = 45.4: the model’s foam height at minute 0, just after pouring, about 45 mm. b = 0.880: each minute about 88\% of the foam remains, so about 12\% collapses.
- a = 201.8697, b = 1.1887, so \hat{y} = 202(1.189)^x, with x in days and \hat{y} the predicted views.
- About 18.9\% a day, since b = 1.189. Over three days 1.189^3 \approx 1.68; the table’s ratios are 1.70, 1.71, 1.64, 1.68, which match.
- A line. The differences are -3.9, -4.1, -3.8, -4.2, all near -4, while the ratios fall from 0.87 to 0.77; a train at steady speed covers the same distance every 10 minutes.
- Exponential. The drops shrink (60, 50, 45, 35) but the ratios are 0.85, 0.853, 0.845, 0.857, nearly constant: the bike loses about 15\% of its value each year.
- Not really. Differences (0.40, 0.40, 0.50, 0.40) and ratios (1.050, 1.048, 1.057, 1.043) are both nearly constant, and over five years both fits stay close to the data. Context decides: prices usually rise by a percent of the current price, which suggests an exponential, and the two models only part ways over many more years.
- The gaps above the line are positive and those below are negative, so they cancel. Every line through (\bar{x}, \bar{y}) has gaps totaling 0, however steep or flat, so the plain total cannot tell good lines from bad. Squares are never negative, so they cannot cancel, and a large miss counts heavily.
- Technology fits an exponential to any positive data, whether or not the data grow by ratios. Whether the family is right comes from the plot (does it bend, with no pattern of misses?) and the context (does the quantity change by a share of itself?).
- The fit uses every point at once and keeps all the misses small together. The ratios in the data wobble around a common value, and the fitted b is a compromise among them, so it need not equal any single one.
- The curve is chosen to fit all six points together, not to pass through the first one. Here the data fall by a nearly steady amount, so the best curve of the form ab^x runs above the first and last readings and below the middle ones. A start value this far from the first reading is one more sign that the family is wrong here.