Correlation is not causation: lurking and confounding variables
Two things rise and fall together. When does that mean one causes the other, and when is something else pulling both?
Where you are 9 in Bivariate data
Why this exists
A scatter plot or an association in a two-way table shows that two variables go together, and the correlation coefficient r says how tightly a straight-line pattern holds. The next question is nearly always “so does one cause the other?” A seaside shop sells more sunscreen in the weeks it sells more ice cream. Fires with bigger crews do more damage. Should the shop push ice cream to sell sunscreen, and should the fire chief send fewer firefighters?
The patterns in those data are real. The conclusions are not. This lesson is about the gap between “these go together” and “this one makes that one happen”, and about reading a claim and saying exactly what its data can and cannot support.
Definition
Two variables are associated when knowing the value of one tells you something about the value of the other; when the pattern is a straight line, they are correlated. One variable causes a change in another when changing the first, with everything else left as it was, changes the second.
Correlation is not causation. An association, however strong, does not by itself show that either variable causes the other. Four kinds of explanation produce the same pattern between an x and a y, and more than one can be at work at once:
| Explanation | What happens |
|---|---|
| Cause | changing x changes y |
| Reverse direction | changing y changes x |
| A third variable | something else changes both x and y |
| Coincidence | the pattern belongs only to this small or hand-picked set of data |
A lurking variable is a variable that is not among those studied but affects both of them, and so creates, hides or changes the association between them.
Two possible causes are confounded when they change together in the data, so that their effects on the response cannot be told apart. A variable that does this, by being associated with both the explanatory variable and the response, is a confounding variable. A lurking variable is a confounding variable that nobody measured.
Intuition
Picture the weather standing behind the plot of sunscreen sales against ice cream sales. A hot week pushes both up; a cool week pulls both down. Neither sale touches the other: stop selling ice cream and the sunscreen shelf would empty just as fast. The association is real, and it will come back next summer. Only its meaning was mistaken.
A lurking variable can also hide a real effect. Figure 1 shows a sandwich shop on twelve days: more staff, more complaints.
Figure 2 marks each day as a weekday or a weekend day. Weekend days are busy, so the owner puts more staff on and more customers means more complaints. Inside each kind of day the pattern runs the other way: extra staff go with fewer complaints. The rising cloud came from the kind of day, not from the staff.
Results
Four explanations. An association between x and y can come from x affecting y, from y affecting x, from a third variable affecting both, or from chance in this one set of data. The numbers alone do not say which.
What a confounder needs. A variable confounds the link between x and y only if it is associated with both. A variable linked to just one of them cannot explain why they go together.
What breaks the link. When chance decides which unit gets which treatment, the groups come out alike, on average, in every other variable, measured or not. Then no other variable travels with the treatment, so a clear difference in the response points to the treatment itself.
Why it is true
The correlation coefficient is built from the pairs and nothing else:
where z_x and z_y are the standardized values. Swap x and y and every product z_x z_y is unchanged, so r is unchanged (Properties of the correlation coefficient): it cannot tell cause from reverse direction. No third variable appears in the formula, so r comes out the same whether or not weather, fire size or the day of the week stands behind both. And any set of points, however it was chosen, has an r, so six points picked by hand can show a strong one. Since the same r fits all four stories, deciding between them needs knowledge of how the data were produced, not more arithmetic.
A confounder needs both links. Suppose a variable w is linked to x but not to y. Sorting the units by w then sorts their x values but leaves their y values unchanged on average, so w cannot make y move with x.
Random assignment works because chance looks at no trait. Every unit, whatever its age, fitness or habits, has the same chance of landing in either group, so with enough units each trait ends up spread about evenly. That is the design idea of an experiment: assigning treatments by chance is what lets it reach conclusions about cause that an observational study cannot.
Worked examples
Ice cream and sunscreen
A seaside shop records eight weeks: the week’s highest temperature in degrees Celsius, the ice creams sold and the bottles of sunscreen sold (made-up data).
| Week | Temp. | Ice creams | Sunscreen |
|---|---|---|---|
| 1 | 14 | 120 | 15 |
| 2 | 19 | 175 | 20 |
| 3 | 17 | 160 | 22 |
| 4 | 22 | 230 | 31 |
| 5 | 26 | 290 | 38 |
| 6 | 24 | 250 | 35 |
| 7 | 31 | 360 | 49 |
| 8 | 28 | 300 | 44 |
Find r for ice creams and sunscreen, and decide whether selling ice cream raises sunscreen sales.
SolutionA calculator gives r \approx 0.99 for the eight pairs (ice creams, sunscreen): a very strong positive linear association.
Now test each explanation. Cause: eating an ice cream does not make anyone buy sunscreen, and nothing in the data shows it does. Reverse direction: buying sunscreen does not make anyone want ice cream either. Coincidence: unlikely, since the pattern holds week after week and has a clear reason. A third variable: the temperature column rises and falls with both sales; the hottest week (week 7) has the most of each and the coolest (week 1) the least. Hot weather sends people to the beach, where they want both.
So the temperature is the lurking variable. The data support “the two sales rise together”; they do not support “selling ice cream raises sunscreen sales”. A promotion on ice cream would be the test of that, and there is every reason to expect it to fail.
Firefighters and damage
For six fires a city recorded the firefighters sent and the damage in thousands of dollars (made-up data): (4, 12), (6, 30), (9, 25), (12, 70), (18, 90), (24, 160). Someone proposes sending fewer firefighters to cut damage. Respond.
SolutionA calculator gives r \approx 0.97, so the association is strong and positive. The proposal assumes cause: that firefighters produce damage. Nothing they do burns buildings; their work is to put fires out.
The likelier story is a third variable: the size of the fire. A large fire does more damage, and the control room sends more firefighters to a large fire. Size drives both columns, so the two rise together even if every firefighter reduces damage.
To judge whether crews help, compare fires of about the same size: among fires of equal size, do bigger crews go with less damage? The six pairs give no information on size, so they cannot answer that. They support “bigger fires get bigger crews and do more damage”, nothing more.
Coloring in the lurking variable
Figures 1 and 2 show the sandwich shop’s twelve days (made-up data). Weekdays: (3, 6), (3, 5), (4, 5), (5, 3), (5, 4), (6, 2). Weekend days: (8, 12), (8, 11), (9, 10), (10, 9), (11, 7), (11, 8). Find r for all twelve days and for each kind of day, and say what the owner should conclude.
SolutionFor all twelve pairs a calculator gives r \approx 0.59: a moderate positive association. For the six weekdays alone r \approx -0.93, and for the six weekend days r \approx -0.97. Both are strong and negative.
The kind of day is a lurking variable. It is linked to staff (weekends get 8 to 11, weekdays 3 to 6) and to complaints (weekends bring more customers, so more chances to complain). That is what Figure 1 picked up. Holding the kind of day fixed, the pattern reverses: more staff, fewer complaints.
The owner should not cut staff to cut complaints. The data are still observational, so even the negative trend inside each group is an association, not proof, but it is far more to the point than the pooled plot. A trend that reverses when the data are split by a lurking variable is called Simpson’s paradox.
A confounded study
A youth soccer club tries new ankle supports. The Saturday squad, which trains on grass, wears them all season; the Wednesday squad, which trains on a hard artificial field, plays without them. Of 40 Saturday players, 4 are injured; of 40 Wednesday players, 10 are. Do the ankle supports prevent injuries?
SolutionThe injury rates are \displaystyle \frac{4}{40} = 10\% and \displaystyle \frac{10}{40} = 25\%, a large gap. But the supports are not the only difference between the squads. The surface differs too, and a hard field can cause injuries of its own; so may the day, the coach or the players’ ages. The supports and the surface changed together, so they are confounded: the data cannot say how much of the gap, if any, the supports made.
A better design mixes the squads: within each squad, chance decides which half of the players wears the supports. Then each surface has players with and without them, the two groups are alike on average in everything else, and a gap in injuries points to the supports. That is Random assignment and confounding.
A confounded two-way table
Quirin records 200 school days: whether he carried an umbrella, and whether his train was late.
| Late | On time | Total | |
|---|---|---|---|
| Umbrella | 21 | 39 | 60 |
| No umbrella | 17 | 123 | 140 |
Does carrying an umbrella make his train late? Then use his split by weather.
| Late | On time | Total | |
|---|---|---|---|
| Rainy, umbrella | 20 | 30 | 50 |
| Rainy, no umbrella | 4 | 6 | 10 |
| Dry, umbrella | 1 | 9 | 10 |
| Dry, no umbrella | 13 | 117 | 130 |
Using row percentages: with an umbrella the train was late on \displaystyle \frac{21}{60} = 35\% of days, without one on \displaystyle \frac{17}{140} \approx 12.1\%. The rows differ a lot, so umbrella and lateness are associated.
Rain is the obvious third variable: it makes him take the umbrella and it slows trains. The split table tests that. On rainy days the late rate is \displaystyle \frac{20}{50} = 40\% with an umbrella and \displaystyle \frac{4}{10} = 40\% without. On dry days it is \displaystyle \frac{1}{10} = 10\% and \displaystyle \frac{13}{130} = 10\%. Once the weather is held fixed the umbrella makes no difference. The pooled gap came from umbrella days being mostly rainy days.
Reading headlines
For each headline, say what the data can support and give a plausible explanation other than the one claimed.
(a) “Shops with more security cameras report more thefts.” (b) “Towns with more libraries hand out more parking tickets.” (c) “Students who sit at the front of the class get higher marks.”
SolutionEach rests on an observational association, so each supports “these go together” and no more.
(a) Reverse direction: a shop that has suffered thefts installs cameras. Cameras may even reduce theft, while the shops that have them still report more.
(b) A third variable, the size of the town: more people means more libraries and more cars to ticket.
(c) Here a cause is plausible, since the front may be easier to hear and see from. But a lurking variable fits too: keen students may choose the front and also work harder. The data cannot separate the two. An experiment that seats students by chance could.
Boundaries
“Correlation is not causation” is a statement about evidence, not a claim that cause is absent. Many associations are causal; the point is that the association alone does not show it.
No association does not prove no cause. A lurking variable can hide or reverse a real effect, as in Figure 1, and r measures only straight-line patterns.
Splitting the data by a variable deals with that variable only. An observational study can split by every variable it measured and still be confounded by one it did not.
Random assignment supports cause only for the units in the experiment and the treatments tried. Carrying the result to a wider population is a separate question, answered by how the units were chosen: Scope of inference: random selection and random assignment.
Where this goes next
Everything here asks how the data were produced. Random assignment and confounding builds the tool that breaks confounding, and Scope of inference: random selection and random assignment separates the two questions a study answers: did the treatment cause the change, and for whom does it hold. Later, a test for a slope asks whether a pattern in a sample could be chance alone. That answers the coincidence question, and none of the other three.
Exercises
Choosing an explanation
For each association, name the most plausible explanation (cause, reverse direction, a third variable, or coincidence), and name the third variable where there is one.
- In a town, weeks with more hot drinks sold at the café are weeks with more colds reported at the school.
- Across a country’s towns, the number of dentists and the number of fillings done each year are strongly associated.
- At a school, students who see the math tutor most often have the lowest marks.
- Over six years, the number of letters in the winning word of a city’s spelling contest and the city’s rainfall in millimeters were (9, 610), (11, 700), (10, 640), (13, 780), (12, 690), (14, 820). Find r and choose an explanation.
- Over a winter, the hours a home’s electric heater runs each day and the electricity the home uses that day.
Lurking variables in data
- Use the table in Example 1 to find r for temperature and sunscreen bottles sold. What does it add to the example?
- A survey of 100 households finds that 18 of the 40 with a home alarm had a break-in in the past five years, against 9 of the 60 without. Find both percentages. Do alarms attract burglars?
- Sieglinde grows seedlings in 80 trays (made-up data): on the sunny windowsill, 24 of 30 rainwater trays and 8 of 10 tap-water trays grew well; on the shady one, 3 of 10 rainwater trays and 9 of 30 tap-water trays did. Find the percentage that grew well for rainwater and for tap water, ignoring the windowsill. Then find it for each water on each windowsill. Does rainwater help?
- For Figure 2, find the mean number of staff and the mean number of complaints on weekdays and on weekend days. Explain how these produce the rising cloud of Figure 1.
Headlines and studies
- “Children who eat dinner with their family get better marks.” Say what the data can support and name a plausible lurking variable.
- A city fits new streetlights on the streets whose residents sent in a petition. Crime falls more on those streets than elsewhere. Explain why the streetlights and something else are confounded.
- A player points to four matches, all won, in which he wore his lucky socks, as proof that the socks help. Which explanation fits best, and what data would you ask for?
Reasoning
- A claim reads: “r = 0.98 is so strong that it must be cause.” Explain why a stronger r does not make cause any more certain.
- In Example 4, explain why choosing by chance, within each squad, which players wear the ankle supports removes the surface as a confounder, while letting the coaches choose would not.
- In the soccer study, suppose the players’ heights differ between the two squads but height has no link to injuries. Explain why height is not a confounding variable.
Answers
- A third variable: cold weather sends people to hot drinks and also comes with more colds.
- A third variable: the town’s population. More people need more dentists and have more teeth to fill.
- Reverse direction: students who are struggling are the ones sent to, or who seek out, the tutor. Low marks lead to tutoring, not the other way round.
- r \approx 0.97. Coincidence: six points, and no way for a spelling word to change the weather or the weather to choose the word.
- Cause: a running heater draws electricity, and the meter counts it. Cold weather is behind the heater hours, but it acts on the electricity through the heater.
- r \approx 0.99. Temperature is strongly linked to sunscreen sales, as it is to ice cream sales, which is what a lurking variable behind both must be.
- \displaystyle \frac{18}{40} = 45\% with an alarm, \displaystyle \frac{9}{60} = 15\% without. No: the data are associational. Reverse direction fits well (a household fits an alarm after a break-in), and so does a third variable (living where break-ins are common leads people to fit alarms).
- Ignoring the windowsill: rainwater \displaystyle \frac{27}{40} = 67.5\%, tap \displaystyle \frac{17}{40} = 42.5\%. Sunny: rainwater \displaystyle \frac{24}{30} = 80\%, tap \displaystyle \frac{8}{10} = 80\%. Shady: rainwater \displaystyle \frac{3}{10} = 30\%, tap \displaystyle \frac{9}{30} = 30\%. No: on each windowsill the two waters do equally well. Rainwater trays were mostly on the sunny sill, which is the real difference.
- Weekdays: mean staff \displaystyle \frac{26}{6} \approx 4.3, mean complaints \displaystyle \frac{25}{6} \approx 4.2. Weekend days: mean staff \displaystyle \frac{57}{6} = 9.5, mean complaints \displaystyle \frac{57}{6} = 9.5. Weekend days sit high on both axes, so the pooled cloud climbs from the weekday group to the weekend group, even though each group falls.
- It supports “children who eat with their family tend to have better marks”. A lurking variable: how settled the home routine is (or how much time parents have), which could lead to both family dinners and steady homework.
- The petition streets differ in more than their lights. Residents organized enough to petition may also watch out for each other and report crime. The lights and the residents’ organization arrive together, so their effects on crime cannot be told apart.
- Coincidence in a hand-picked set: four chosen matches. Ask for every match: the win rate with the socks and without them.
- r measures how tightly the points follow a line, not why. A third variable can produce an r near 1 (Example 1 has r \approx 0.99 with no cause at all), and the formula is the same in either direction, so no value of r separates the four explanations.
- Chance pays no attention to the surface, so each squad’s field gets players with and without the supports in about equal numbers and the surface no longer travels with the supports. Coaches might choose, say, players with past ankle trouble, who differ in injury risk, putting a new difference between the groups.
- A confounder must be associated with both the treatment and the response. Height differs between the squads, so it is linked to the supports, but if it has no link to injuries it cannot produce a difference in injury rates.