The law of large numbers and the gambler's fallacy
A fair coin has just landed heads five times running. Is tails now more likely? And if it is not, how do heads and tails still come out about even in the long run?
Where you are 14 in Basic probability
Why this exists
Experimental probability and what randomness means both showed a running share jumping about at first and then settling. Two questions were left open. Settling near what? And what does the settling say about the next trial?
The second question has a famous wrong answer. After a long run of black on a wheel, someone bets on red because red is “due”. After four daughters, a relative says the next child is surely a son. A lottery number not drawn for two years gets called “overdue”. Each time the idea is that chance must balance its books.
The settling itself has been tested directly. During the Second World War a mathematician held prisoner passed the time by tossing a coin 10{,}000 times, recording every result, and published an account afterwards; the share of heads wandered early on and ended close to one half. This lesson says exactly what that settling is, and why it gives no support at all to “due”.
Definition
Trials of a chance process are independent when the result of one has no effect on the chances in any other, as in independent events: spins of a wheel, tosses of a coin, draws from a bag with the counter put back each time.
The law of large numbers. Repeat a chance process in independent trials under the same conditions, and let p be the probability of one outcome. As the number of trials grows, the relative frequency of that outcome tends to get closer to p: with many trials, landing far from p becomes very unlikely. The law is about the share, \displaystyle \frac{\text{count}}{n}, never the count itself.
The gambler’s fallacy is the belief that, in independent trials, a run of one outcome makes a different outcome more likely next time, as though it were “due”. It is false. Independent trials have no memory: the probability of each outcome on the next trial is still p, whatever came before.
Intuition
Quillan runs a prize wheel at a fair. It has 10 equal sections, 4 red, 4 black and 2 green, so \displaystyle P(\text{red}) = \frac{4}{10} = 0.4 on every spin. He keeps a running count of reds.
| Spins | Reds | Share of red |
|---|---|---|
| 10 | 6 | 0.6 |
| 50 | 17 | 0.34 |
| 100 | 37 | 0.37 |
| 500 | 207 | 0.414 |
| 1{,}000 | 412 | 0.412 |
| 5{,}000 | 2{,}033 | 0.4066 |
The share falls by 0.26 between the first two rows and by about five thousandths between the last two.
Now ask the gambler’s question. After 5{,}000 spins the share is a little above 0.4. Is red less likely on the next spin, to pull it back? No. The wheel has no idea what the share is; its sections are the same size as on spin 1, so P(\text{red}) = 0.4 again. Each new batch of spins comes out near 40\% red on its own account, and in time the new spins simply outnumber the early ones. An early excess of reds is never paid back. It is diluted.
Results
The law of large numbers. For independent trials under the same conditions, the relative frequency of an outcome with probability p tends toward p as the number of trials grows.
No memory. For independent trials, the probability of an outcome on the next trial is p, whatever the earlier results were, including any run of the same outcome.
Shares and counts. If an outcome with probability p has happened c times in n trials, its expected count is pn and its gap is c - pn. Then
The share’s distance from p is the gap divided by the number of trials. So the share can close in on p while the gap in the counts grows, as long as n grows faster than the gap. This works for any p, not only the fair coin’s \displaystyle \frac{1}{2}.
Why it is true
Shares and counts. Write p as \displaystyle \frac{pn}{n}, so both terms have the denominator n:
No memory. For independent trials this is what independence means. The multiplication rule makes it concrete: before five tosses of a fair coin, HHHHT and HHHHH each have probability \displaystyle \left(\frac{1}{2}\right)^5. Once four heads have happened, these are the only two ways left, equally likely, so each now has probability \displaystyle \frac{1}{2}. The unlikely run of four is used up once it has happened (Example 2 works this through).
Why the share settles. A full proof needs tools from later courses. What randomness means shows by counting why it is believable: among all the equally likely lists of n coin tosses, almost all have a share of heads near one half once n is large.
Worked examples
Counts drift while the share settles
Use Quillan’s table. At each stop, find the expected count of reds and the gap. Then use the gap to find how far the share is from 0.4 after 5{,}000 spins, and compare with the table.
SolutionThe expected count is p \times n = 0.4n, and the gap is the count minus that.
| Spins n | Expected 0.4n | Reds | Gap |
|---|---|---|---|
| 10 | 4 | 6 | +2 |
| 50 | 20 | 17 | -3 |
| 100 | 40 | 37 | -3 |
| 500 | 200 | 207 | +7 |
| 1{,}000 | 400 | 412 | +12 |
| 5{,}000 | 2{,}000 | 2{,}033 | +33 |
After 5{,}000 spins the gap is 33, so the share is \displaystyle \frac{33}{5{,}000} = 0.0066 above 0.4, that is 0.4066. The table agrees: \displaystyle \frac{2{,}033}{5{,}000} = 0.4066.
The gap grew from 2 to 33, while the share’s distance from 0.4 shrank from \displaystyle \frac{2}{10} = 0.2 to 0.0066. Nothing pulled the counts together; the number of spins simply grew far faster than the gap.
Four daughters
Nephele and Orrin have four daughters and are expecting a fifth child. A relative says it is surely a boy this time. Use the model in which each child is a girl or a boy with probability \displaystyle \frac{1}{2}, independently.
(a) What is the probability that the fifth child is a boy? (b) Before any of the children were born, what was the probability of four girls then a boy? Of five girls? (c) Explain why (b) gives the relative no support.
Solution(a) The children are independent in this model, so \displaystyle P(\text{boy}) = \frac{1}{2}.
(b) By the multiplication rule each order of five children has probability \displaystyle \left(\frac{1}{2}\right)^5 = \frac{1}{32}, so both are \displaystyle \frac{1}{32}.
(c) Five girls was unlikely before the family started, but four girls then a boy was exactly as unlikely. With four girls already here, those two orders are all that is left, with equal chances. The unlikely part, four girls, with probability \displaystyle \frac{1}{16}, has already happened. The model simplifies real families, but within it the answer is exact.
A head start is diluted, not paid back
Orrin tosses a fair coin and the first 10 tosses all land heads. He keeps going, and suppose every later toss comes out exactly as expected, half heads.
(a) How many heads, and what share, after 1{,}000 tosses? After 10{,}000? (b) How many more heads than tails are there at each point? (c) What does this say about how the share returns toward \displaystyle \frac{1}{2}?
Solution(a) After 1{,}000 tosses there are 990 later ones, with \displaystyle \frac{1}{2} \times 990 = 495 heads, so 10 + 495 = 505 heads and a share of \displaystyle \frac{505}{1{,}000} = 0.505. After 10{,}000 there are 9{,}990 later tosses with 4{,}995 heads, so 5{,}005 heads and a share of \displaystyle \frac{5{,}005}{10{,}000} = 0.5005.
(b) 505 against 495, and 5{,}005 against 4{,}995: 10 more heads both times.
(c) The 10 extra heads are never canceled, since nothing in later tosses favors tails. The share returns because they become a smaller and smaller part of the whole. In the identity, the gap is 505 - 500 = 5 and \displaystyle \frac{5}{1{,}000} = 0.005; later it is \displaystyle \frac{5}{10{,}000} = 0.0005. Real tosses would not split exactly in half, but the point stands: the early excess is diluted.
An overdue lottery number
A town’s weekly draw takes one ball from 50 balls numbered 1 to 50, all equally likely, and the ball goes back in afterwards. Number 17 has not been drawn for 120 weeks, and Rosalba says it is overdue.
(a) What is the probability that 17 is drawn next week? (b) How many times would 17 be expected in 120 draws? (c) What was the chance, before those weeks, of 17 missing all 120 draws? (d) Should Rosalba choose 17?
Solution(a) The same 50 balls are used each week, so the draws are independent: \displaystyle P(17) = \frac{1}{50} = 0.02.
(b) 0.02 \times 120 = 2.4, so about two or three times.
(c) Each week 17 is missed with probability \displaystyle \frac{49}{50} = 0.98, and by the multiplication rule
about 9 in 100: uncommon, but nothing extraordinary.
(d) There is no reason for or against. Next week 17 has the same 0.02 as every other number. The balls carry no record of the past 120 weeks.
Streaks at the dartboard and in the sky
Petronella hits the inner ring with about half her darts. In one session she hits five in a row, and her friends say she is “hot”.
(a) If her throws were independent, each a hit with probability \displaystyle \frac{1}{2}, what is the chance that five particular throws are all hits, and of a hit on the throw after the run? (b) Here are 20 throws from one session, H for a hit and M for a miss: M H H M H H H H H M M H M M H M H H M M. Find the share of hits, and count the stretches of five throws in a row. How many of them are all hits? (c) It has rained 8 days running, and Mortimer says a dry day is due. Does the no-memory rule apply?
Solution(a) \displaystyle \left(\frac{1}{2}\right)^5 = \frac{1}{32} by the multiplication rule; the next throw is still \displaystyle \frac{1}{2}.
(b) There are 11 hits, a share of \displaystyle \frac{11}{20} = 0.55, an ordinary session. A stretch of five can start at throw 1, 2, and so on up to 16, so there are 16 stretches, and one of them, throws 5 to 9, is all hits. Five throws named in advance are all hits with probability only \displaystyle \frac{1}{32}, but a session gives a run 16 overlapping places to appear, so a run somewhere is far more likely: a count of all 2^{20} equally likely sessions shows about one in four contains one. Independent trials produce streaks without any cause. Whether a real player’s throws are independent is a question for data, since confidence or tiredness could link them.
(c) No, because days of weather are not independent: a wet weather system often stays for several days, so if anything a rainy day makes another more likely. Mortimer is still wrong, since nothing makes the weather balance out.
Due, or rigged?
A neighboring stall has a wheel said to be like Quillan’s, with 4 of its 10 sections black. Its board shows the last ten spins: R, B, G, B, R, B, G, B, B, B. Mortimer bets on red because red is due. Rosalba wonders whether the wheel favors black.
(a) If the wheel is as claimed, what is the probability of red next? (b) Do these ten spins give strong evidence that the wheel favors black? (c) The stall’s record for the season shows 920 black in 2{,}000 spins. What now? (d) Whose reasoning holds up?
Solution(a) P(\text{red}) = 0.4, whatever the board shows.
(b) Black came up 6 times against an expected 0.4 \times 10 = 4, a gap of 2. In ten spins one spin is worth 0.1 of the share, and Quillan’s own fair wheel was 0.2 off after ten spins, so a gap of two is ordinary. Not strong evidence.
(c) Expected 0.4 \times 2{,}000 = 800, so the gap is 920 - 800 = 120 and the share is \displaystyle \frac{920}{2{,}000} = 0.46, which is \displaystyle \frac{120}{2{,}000} = 0.06 above 0.4. Quillan’s wheel, which really is 0.4, stayed within 0.015 of 0.4 from 500 spins on. A share 0.06 off after 2{,}000 spins is strong evidence that the wheel favors black, with P(\text{black}) \approx 0.46 the better estimate.
(d) Rosalba’s. Past spins are evidence about what the wheel is, that is, about p; never about what is due. If the wheel favors black, black is more likely next, the opposite of Mortimer’s bet.
Boundaries
Independent trials under the same conditions. When counters are drawn from a bag without being put back, a run of red does make blue more likely, because the bag has changed: these are dependent events, and that is arithmetic, not the fallacy. Weather, a wheel that wears unevenly or a player who tires are not repeats of one fixed process either.
A tendency, never a guarantee. For any fixed number of trials the share may be some way from p; that only becomes very rare. The gap in the counts can grow without limit while the share settles, and the law says nothing about any single trial.
When p is unknown, the past is evidence about p. Using a long record to estimate or doubt a probability is the law used correctly (Example 6). What the past cannot do, in independent trials, is change the probability of the next result.
Where this goes next
The law is why running a simulation and estimating a probability works: a well-designed simulation run many times gives a relative frequency close to the true probability, and probability models supply the p to compare it with.
The same settling holds for averages: over many plays of a game the average result per play settles near the expected value, which is why a game that slightly favors a stall wins for the stall over a season.
Exercises
Shares and counts
- A spinner lands blue with probability 0.25. It lands blue 13 times in 40 spins, 94 in 400 and 1{,}018 in 4{,}000. At each stop find the share, the expected count, the gap, and the share’s distance from 0.25.
- A fair cube is rolled 600 times and shows 6 on 112 of them. Find the expected count, the gap, the share of sixes to three decimal places, and the share’s distance from \displaystyle \frac{1}{6}.
- A fair coin is tossed 3{,}000 times and the share of heads is 0.508. How many heads and tails were there, and how far is the count of heads from the expected count?
- An outcome has probability 0.3. Run A has 18 successes in 50 trials; run B has 630 in 2{,}000. Which share is closer to 0.3, and which gap is larger?
- In 9{,}000 trials of an outcome with probability 0.5, the count is 45 above the expected count. Find the count and the share.
No memory
- A fair coin lands heads 6 times in a row. What is the probability of heads next? Before any tossing, what were the probabilities of six heads then a tail, and of seven heads?
- On Quillan’s wheel, green comes up three spins in a row. What is the probability of green on the next spin? Before those spins, what was the probability of three greens in a row?
Independent or not
- Say whether the trials are independent, and whether “due” reasoning is the gambler’s fallacy: (a) spins of a fair wheel; (b) counters drawn from a bag without being put back; (c) rainy and dry days in a row; (d) a weekly draw with the ball put back.
- A bag holds 5 red and 5 blue counters. Four are drawn without being put back, all red. Find the probabilities that the next is red and that it is blue, and explain why “blue is more likely now” is not the gambler’s fallacy here.
- A spinner lands yellow with probability 0.8, and its spins are independent. What is the chance that four given spins all land yellow? After four yellows in a row, what is the chance of yellow next? After four spins that were not yellow?
Reasoning
- Someone says: “If you toss a coin long enough, the numbers of heads and tails will come out equal.” Correct this.
- Petronella says: “The share of heads must come back to \displaystyle \frac{1}{2}, so after a run of heads, tails has to come more often.” What is wrong, and how does the share come back?
- Explain why, for a fair coin, ten heads in a row is exactly as likely as HTTHHTHTTH, and find that probability.
- A coin said to be fair lands heads 20 times in a row. Mortimer says tails is due; Rosalba says the coin may not be fair. Which of them is reasoning mathematically?
- Mortimer says: “It rains here on about a third of days, and it has been dry for 9 days, so by the law of large numbers rain is due.” Give two separate reasons the argument fails.
Answers
- \displaystyle \frac{13}{40} = 0.325, \displaystyle \frac{94}{400} = 0.235, \displaystyle \frac{1{,}018}{4{,}000} = 0.2545. Expected 10, 100, 1{,}000; gaps +3, -6, +18. Distances 0.075, 0.015, 0.0045: the gap grew while the share closed in.
- Expected \displaystyle \frac{1}{6} \times 600 = 100; gap 112 - 100 = 12; share \displaystyle \frac{112}{600} \approx 0.187; distance \displaystyle \frac{12}{600} = 0.02.
- Heads 0.508 \times 3{,}000 = 1{,}524, tails 3{,}000 - 1{,}524 = 1{,}476. Expected heads 1{,}500, so heads are 24 above it.
- A: \displaystyle \frac{18}{50} = 0.36, expected 15, gap +3. B: \displaystyle \frac{630}{2{,}000} = 0.315, expected 600, gap +30. B’s share is closer (0.015 away against 0.06), though its gap is ten times larger.
- Expected 4{,}500, so the count is 4{,}545 and the share is \displaystyle \frac{4{,}545}{9{,}000} = 0.505.
- \displaystyle \frac{1}{2}. Each sequence of seven independent tosses has \displaystyle \left(\frac{1}{2}\right)^7 = \frac{1}{128}.
- \displaystyle P(\text{green}) = \frac{2}{10} = 0.2 on the next spin, as on every spin. Three greens: 0.2^3 = 0.008.
- (a) Independent; “due” is the fallacy. (b) Not independent: each draw changes the bag, so the chances really shift, and that is not the fallacy. (c) Not independent, since weather comes in spells; “due” is still wrong, as nothing balances the weather. (d) Independent; the fallacy.
- Six counters remain, 1 red and 5 blue: \displaystyle P(\text{red}) = \frac{1}{6}, \displaystyle P(\text{blue}) = \frac{5}{6}. The draws are dependent: the bag holds fewer reds now, so blue really is more likely. The fallacy is claiming this for independent trials, where nothing changes.
- 0.8^4 = 0.4096. After four yellows: 0.8. After four that were not yellow: 0.8.
- The law is about the share, not the counts. The share of heads tends toward \displaystyle \frac{1}{2}, but the difference between the numbers of heads and tails need not shrink and can grow; it only becomes small compared with the number of tosses.
- Independent tosses have no memory, so tails is never more likely than \displaystyle \frac{1}{2}. The share comes back by dilution: later tosses, about half heads, outnumber the early run, which counts for less and less.
- Each toss is \displaystyle \frac{1}{2} for either face, independently, so any one particular sequence of ten tosses has probability \displaystyle \left(\frac{1}{2}\right)^{10} = \frac{1}{1{,}024}. Ten heads is one particular sequence, just as HTTHHTHTTH is.
- Rosalba. If the coin is fair, tails is still \displaystyle \frac{1}{2}, so Mortimer is wrong either way. But 20 heads from a fair coin has probability \displaystyle \left(\frac{1}{2}\right)^{20}, less than one in a million, so the run is good evidence the coin is not fair, and then heads is more likely next.
- Days of weather are not independent trials, so the law and the no-memory rule do not apply as stated. And even for independent trials, the law is about the share over many trials; it never makes one outcome more likely next, so nothing is ever “due”.