The central limit theorem
Order totals at a food co-op are lopsided, mostly small with a few big ones. Why would the mean of 36 of them follow a bell curve, and how far can that be trusted?
Where you are 8 in Distributions
Why this exists
A sampling distribution is the pattern a statistic makes over many random samples. Simulation can draw one, but only for a population that is fully known, and a real study knows its population only through its one sample. To say how far a sample mean is likely to sit from the population mean, a study needs the shape of the sampling distribution without seeing the population.
The central limit theorem supplies it. Whatever shape the population has, skewed, lumpy or flat, the mean of a large enough random sample is approximately normal, and the normal curve’s areas from Normal calculations: areas and percentiles then answer questions about it. That is why the normal curve runs through nearly every inference method for means.
The lesson runs over two days. Day 1 is what the theorem says and how to use it. Day 2 is what it needs, and how large “large enough” has to be. The co-op, its people and every number in it are made up.
Definition
Take n independent random draws X_1, X_2, \ldots, X_n from one population with mean \mu and a finite standard deviation \sigma, and form the sample mean
The central limit theorem. As n grows, the sampling distribution of \bar{x} gets closer and closer to a normal distribution with mean \mu and standard deviation \displaystyle \sigma/\sqrt{n}:
where \approx here means “has approximately the distribution”.
In standardized form: the standard normal areas give the chances for
more and more accurately as n grows.
The statement has three parts, and they are not equally strong.
- Center. \mu_{\bar{x}} = \mu for every n, exactly. The sample mean is aimed at the population mean (unbiased).
- Spread. \displaystyle \sigma_{\bar{x}} = \sigma/\sqrt{n} for every n, exactly, given independent draws.
- Shape. Approximately normal, with the approximation improving as n grows. This is the part the theorem adds, and it is the only approximate part.
The theorem never names a sample size. It says the error shrinks toward zero; how big n must be for the error to be small depends on the population.
Intuition
Figure 1 is the co-op’s population: one order total, most between 5 and 25 dollars, with a long tail of big weekly shops.
Averaging two orders already changes the picture. A big order is rare, so two big ones together are rarer still; a big order is usually paired with an ordinary one, and their mean lands in the middle. Every added order pulls the mean further toward 18 from both sides, and the tail that one order can make is diluted by the others. Figure 2 shows the result: at n = 2 the lopsided shape is still there, at n = 10 it is faint, and at n = 36 the histogram sits almost on the dashed normal curve.
Two things happen at once and should be kept apart. The histograms narrow, because \displaystyle \sigma/\sqrt{n} shrinks; that is true even for tiny n. And they turn normal, which is gradual and is the theorem’s real content.
Results
Day 1: the shape of the sample mean
For independent draws from any population with mean \mu and finite standard deviation \sigma:
and for large n the distribution of \bar{x} is approximately normal, so
If the population is itself normal, \bar{x} is exactly normal for every n, with no approximation.
The same holds for the total X_1 + \cdots + X_n = n\bar{x}, which has mean n\mu and standard deviation \displaystyle \sigma\sqrt{n} and is approximately normal for large n.
Day 2: what it needs
Independent draws (a random sample, and under the ten percent condition when sampling without replacement), and a finite \sigma. The more skewed or lumpy the population, the larger n must be before the normal shape is a good fit.
Why it is true
Center and spread. The mean of a sum is the sum of the means, so \bar{x} has mean \displaystyle \frac{1}{n}(n\mu) = \mu. For independent draws, variances add, so the sum has variance n\sigma^2, and dividing by n divides the variance by n^2:
Shape. A full proof uses tools from a later course in probability. One piece of it can be seen here. The usual measure of lopsidedness, the skewness, is the mean of the cubed z-scores: 0 for a symmetric shape, positive for a right tail. For independent draws, the third powers of deviations add the way variances do, and working that through shows that the skewness of \bar{x} is the population’s skewness divided by \displaystyle \sqrt{n}. The co-op model has skewness about 1.33, so the mean of 36 orders has skewness about 1.33/6 \approx 0.22, close to the 0 of a normal curve. The finer measures of the population’s shape fade too, and faster, which is why the limit is the same bell curve whatever the starting shape. It also shows why a more skewed population needs a larger n: the skewness starts higher and shrinks only like \displaystyle 1/\sqrt{n}.
Worked examples
Normal areas use z rounded to two decimal places and \Phi(z), the area to the left under N(0, 1), as in Normal calculations: areas and percentiles. Values used here:
| z | \Phi(z) | z | \Phi(z) |
|---|---|---|---|
| 0.78 | 0.7823 | 1.50 | 0.9332 |
| 0.83 | 0.7967 | 1.645 | 0.9500 |
| 1.00 | 0.8413 | 1.96 | 0.9750 |
| 1.03 | 0.8485 | 2.00 | 0.9772 |
| 1.18 | 0.8810 | 2.24 | 0.9875 |
| 1.20 | 0.8849 | 2.50 | 0.9938 |
For a negative z, \Phi(-z) = 1 - \Phi(z).
Day 1: reading the four histograms
Nimal had a computer draw 1000 random samples of each size from the model in Figure 1 and record each sample’s mean. Figure 2 shows the results; the table summarizes them.
| n | mean of the 1000 means | SD of the 1000 means |
|---|---|---|
| 2 | 17.97 | 9.08 |
| 5 | 18.05 | 5.24 |
| 10 | 18.08 | 3.72 |
| 36 | 18.00 | 2.02 |
Compare the centers and spreads with the theorem, describe how the shape changes, and judge the normal curve at n = 36, where 958 of the 1000 means lay within 4 dollars of 18.
SolutionCenter. All four means sit within 0.1 dollar of \mu = 18, as \mu_{\bar{x}} = \mu says. One run of 1000 samples lands near the theorem’s value, not on it.
Spread. The theorem predicts \displaystyle \sigma/\sqrt{n}:
Each simulated spread is within about 0.6 dollar of its prediction. These two facts hold at every sample size; they come from the rules for means and variances, not from the theorem’s limit.
Shape. At n = 2 the histogram keeps the population’s long right tail, peaking left of 18 and running past 60. At n = 5 and n = 10 the tail shortens and the peak moves toward the center. At n = 36 the bars form a single, nearly even hump from about 12 to 26 dollars, close to the dashed curve.
The check at n = 36. Four dollars is 2\sigma_{\bar{x}}. A normal curve puts \Phi(2) - \Phi(-2) = 0.9772 - 0.0228 = 0.9544 of its area within 2 standard deviations, and the run gave 958/1000 = 0.958. The normal model describes the mean of 36 orders well, although a single order is nothing like normal.
Day 1: a normal probability for the mean of 36 orders
Chamari will record the next 36 orders, a random sample of the co-op’s orders. Use the theorem to find the chance that their mean is more than 21 dollars, and compare with Nimal’s simulation, in which 69 of 1000 means were above 21.
SolutionThe theorem gives the mean and standard deviation of \bar{x}, and n = 36 is large enough here for its normal shape (Example 1 checked it):
Standardize 21 against the distribution of \bar{x}, not of one order:
so
About 7 samples of 36 in 100 have a mean above 21 dollars. The simulation gave 69/1000 = 0.069, in agreement. Because the model in Figure 1 is fully known, a computer can also find the exact chance, 0.0721. The normal answer is a little low: the means of 36 still carry a slight right tail (skewness about 0.22), so large means are a little more common than the bell curve says. For most purposes, 0.067 against 0.072 is close.
Day 1: the theorem is about the mean, not the orders
Chamari’s sample of 36 orders arrives. She reasons: “With n = 36, the central limit theorem says things are normal now, so about 2.5\% of orders should be above \mu + 2\sigma = 42 dollars, and about 6.7\% below \mu - 1.5\sigma = 0 dollars.” Explain what is wrong, and find what the model says instead.
SolutionThe theorem describes the distribution of \bar{x} over many samples. It says nothing about the individual orders, whose distribution is the population’s, Figure 1, however many are sampled. A larger sample makes the sample’s histogram look more like the skewed population, not less.
Her second claim shows the error plainly: \displaystyle z = \frac{0 - 18}{12} = -1.5 and \Phi(-1.50) = 0.0668, so a normal model of one order would put 6.7\% of orders below 0 dollars, which is impossible. The real model puts 0 there.
For the upper tail, the model’s exact chance of an order above 42 dollars is 0.0459, about 4.6\%, nearly twice the normal model’s 2.3\%. The long right tail holds more big orders than a bell curve would.
What she can use the normal curve for is the mean of her 36 orders: a mean above \mu + 2\sigma_{\bar{x}} = 22 dollars has chance about 1 - 0.9772 = 0.0228.
Day 2: a small sample from a skewed population
Dilani samples only 5 orders. Find the normal approximation to P(\bar{x} > 30), and compare with the simulation (22 of 1000 means of 5 were above 30) and the model’s exact chance, 0.0246.
SolutionThe center and spread are still exact at n = 5:
so the normal model gives
The simulation gave 0.022 and the exact chance is 0.0246: the normal answer is about half the truth. At n = 5 the skewness of \bar{x} is still \displaystyle 1.33/\sqrt{5} \approx 0.60, and the second panel of Figure 2 shows the long tail that the normal curve misses. At n = 36 (Example 2) the same kind of question was off by 0.005; at n = 5 the tail chance is off by a factor of two. For this population, 5 orders is not large enough for the normal shape, though the mean 18 and the spread 5.37 are still right.
Day 2: when 30 is not enough
To reward members, the co-op enters every order in a draw: each order, independently, has a 1 in 50 chance of winning a 50 dollar voucher. Let X be an order’s voucher value, 50 or 0. Nimal looks at samples of 30 orders. Find the mean and standard deviation of \bar{x}, and compare the normal model with the exact distribution (Figure 3) for P(\bar{x} < 0) and P(\bar{x} \ge 5).
SolutionOne order. \mu = 50(0.02) = 1 dollar. The variance is 50^2(0.02)(0.98) = 49, so \sigma = 7 dollars.
The mean of 30. \mu_{\bar{x}} = 1 and
The exact distribution. The number of winners W among 30 orders is binomial with n = 30, p = 0.02, and \bar{x} = 50W/30. So \bar{x} = 0 when nobody wins, with chance 0.98^{30} \approx 0.5455. One winner gives \bar{x} \approx 1.67 with chance 0.3340, and two give 3.33 with chance 0.0988.
The normal model against the truth. For P(\bar{x} < 0): \displaystyle z = \frac{0 - 1}{1.278} \approx -0.78, and the normal model gives \Phi(-0.78) = 1 - 0.7823 = 0.2177. The truth is 0, since no mean is negative. For P(\bar{x} \ge 5), three or more winners: exactly,
while the normal model gives \displaystyle z = \frac{5 - 1}{1.278} \approx 3.13 and about 0.0009, some 25 times too small.
Figure 3 shows why. More than half the samples sit on a single value, 0, with a short staircase to the right: nothing like a bell. The population is extremely skewed (its skewness is about 6.9, against 1.33 for order totals) and lumpy (two values only), and at n = 30 the mean’s skewness is still \displaystyle 6.9/\sqrt{30} \approx 1.25, as lopsided as a single co-op order. Here \bar{x} is 50 times a sample proportion of winners, so the guide used for proportions in Sampling distribution of a sample proportion, at least 10 expected winners, applies: 0.02n \ge 10 means n \ge 500 orders. “Thirty” is a rough guide for moderately skewed populations, not part of the theorem.
Day 2: a total instead of a mean
The co-op’s delivery van takes the next 50 orders, a random sample. Find the chance that their total is more than 1000 dollars.
SolutionThe total T = 50\bar{x} is a fixed multiple of the mean, so it is approximately normal whenever \bar{x} is, and 50 orders from this population is past the 36 that Example 1 found to fit well. Its mean and standard deviation, from the sum rules:
Then \displaystyle z = \frac{1000 - 900}{84.85} \approx 1.18 and
The same answer comes from the mean: a total above 1000 is a mean above 20, and \displaystyle \frac{20 - 18}{12/\sqrt{50}} \approx 1.18. About 12 vans in 100 carry more than 1000 dollars of orders. Note that the total’s spread grows with n (like \displaystyle \sqrt{n}) while the mean’s shrinks (like \displaystyle 1/\sqrt{n}); both become normal.
Boundaries
It is about the sample mean, not the data. The individual values keep the population’s shape at every n (Example 3). The theorem also covers sums and, through 0 and 1 values, sample proportions; it does not say that a sample median or a sample standard deviation becomes normal at the same pace.
Independent draws. The draws must not influence one another: a random sample, with the ten percent condition when sampling without replacement. Thirty orders from one catering event, or one shopper’s thirty weekly orders, are linked, and neither \displaystyle \sigma/\sqrt{n} nor the normal shape is promised.
A finite standard deviation. Some theoretical models have tails so heavy that their variance is infinite; for them the mean never settles into a bell curve, however large n. Real data on a bounded scale, like order totals, always have finite \sigma.
Large enough depends on the population. “At least 30” is a rule of thumb for moderately skewed populations, not part of the theorem. A nearly symmetric population is close to normal at n = 10 or less; a normal population needs n = 1; an extremely skewed or lumpy one, like the voucher draw, may need hundreds (Example 5). Tail chances need a larger n than chances near the center.
Conditions for inference come later. Checking whether a given sample is large enough, or the population normal enough, is the work of Sampling distribution of a sample mean.
Where this goes next
The formulas \mu_{\bar{x}} = \mu and \displaystyle \sigma_{\bar{x}} = \sigma/\sqrt{n}, together with the conditions for using a normal model for a mean, are set out in Sampling distribution of a sample mean; the same ideas for a share of yes answers are in Sampling distribution of a sample proportion. Because the theorem works without knowing the population’s shape, it lets a study turn one sample mean into a confidence interval and judge, in Hypothesis testing, whether a sample mean is surprising under a claimed \mu. Random variables with a normal shape in general are the subject of Normal random variables.
Exercises
Use the co-op model (\mu = 18 dollars, \sigma = 12 dollars) unless told otherwise, the \Phi table above, and assume random samples.
Day 1: center and spread of the sample mean
- Find \mu_{\bar{x}} and \sigma_{\bar{x}} for samples of 16 orders.
- Find \sigma_{\bar{x}} for samples of 64 orders, and compare with samples of 16.
- How many orders must a sample have for \sigma_{\bar{x}} to be 1 dollar?
- Find \sigma_{\bar{x}} for samples of 4 orders. Using Figure 2, would a normal model for \bar{x} be trustworthy at this size? Explain.
Day 1: normal probabilities for a mean
- For samples of 64 orders, find P(\bar{x} < 15).
- For samples of 36 orders, find P(16 < \bar{x} < 20).
- For samples of 144 orders, find P(\bar{x} > 19).
- For samples of 36 orders, find the value that the sample mean exceeds only 5\% of the time.
- Dilani’s random sample of 36 orders has a mean of 23 dollars. Find P(\bar{x} \ge 23) if \mu = 18, and say whether her result is surprising.
- Nimal’s random sample of 36 orders has a mean of 18.40 dollars and a largest order of 61 dollars. Is the 61 dollar order a sign that something is wrong? Explain using the theorem.
Day 2: how large is large enough
- A co-op promotion refunds 10 dollars on an order with chance 0.05, independently, and nothing otherwise. For samples of 20 orders, find the mean and standard deviation of the mean refund \bar{x}, the exact chance that \bar{x} = 0, and the normal model’s chance that \bar{x} < 0.
- For the refund in Exercise 11, how many orders give 10 expected refunds, the guide used for proportions?
- Loaves at the co-op’s bakery weigh, independently, a normal amount with mean 500 g and standard deviation 20 g. Find the chance that 4 loaves have a mean above 512 g, and say why n = 4 is enough here.
- Population A is flat: every whole number of dollars from 1 to 35 equally likely. Population B is the co-op’s order totals. Which needs the larger sample before \bar{x} looks normal? Explain.
Day 2: totals
- Find the chance that 100 orders total more than 1900 dollars.
- Find the middle 95\% of totals of 25 orders.
- One order has \sigma = 12 dollars. Find the standard deviation of the total of 9 orders and of their mean, and explain why one grows and the other shrinks.
Reasoning
- A classmate says the theorem requires n \ge 30. Explain what the theorem actually says about sample size.
- Nimal records 30 orders, all placed by one family over the year. Explain why the theorem cannot be used for their mean as a picture of all orders.
- Explain why a sample of 500 orders has a histogram that looks more skewed than a sample of 20, while the sampling distribution of \bar{x} for n = 500 is more normal.
- If the population is already normal, what does the central limit theorem add? Explain.
- Explain why the normal model failed for the voucher draw at n = 30 but worked for order totals at n = 36, using skewness.
Answers
- \mu_{\bar{x}} = 18 dollars, \displaystyle \sigma_{\bar{x}} = \frac{12}{\sqrt{16}} = 3 dollars.
- \displaystyle \frac{12}{\sqrt{64}} = 1.5 dollars, half of 3: four times the sample halves the spread.
- 144 orders: \displaystyle \frac{12}{\sqrt{n}} = 1 gives \displaystyle \sqrt{n} = 12.
- \displaystyle \frac{12}{\sqrt{4}} = 6 dollars. No: at n = 5 Figure 2 still shows a long right tail, and n = 4 is smaller, so normal tail chances would be off (Example 4 found them off by half at n = 5).
- 0.0228: \sigma_{\bar{x}} = 1.5, \displaystyle z = \frac{15 - 18}{1.5} = -2, \Phi(-2.00) = 0.0228.
- 0.6826: \sigma_{\bar{x}} = 2, z = \pm 1, 0.8413 - 0.1587 = 0.6826.
- 0.1587: \sigma_{\bar{x}} = 1, z = 1, 1 - 0.8413 = 0.1587.
- About 21.29 dollars: 18 + 1.645(2) = 21.29.
- 0.0062: \displaystyle z = \frac{23 - 18}{2} = 2.5, 1 - 0.9938 = 0.0062. Surprising: under \mu = 18 only about 6 samples in 1000 have a mean this high.
- No. The theorem is about the mean, and 18.40 is ordinary (z = 0.2). Single orders keep the population’s long right tail, so one large order among 36 is not unusual: the model gives about a 0.21 chance that at least one of 36 orders is above 61 dollars.
- Mean 0.5 dollars; one order has \displaystyle \sigma = 10\sqrt{0.05 \times 0.95} \approx 2.179, so \displaystyle \sigma_{\bar{x}} = \frac{2.179}{\sqrt{20}} \approx 0.487 dollars. Exactly, P(\bar{x} = 0) = 0.95^{20} \approx 0.3585. The normal model gives \displaystyle z = \frac{0 - 0.5}{0.487} \approx -1.03 and \Phi(-1.03) = 0.1515 for an impossible negative mean. Not normal at n = 20.
- 200 orders: 0.05n = 10.
- 0.1151: \displaystyle \sigma_{\bar{x}} = \frac{20}{\sqrt{4}} = 10, z = 1.2, 1 - 0.8849 = 0.1151. The population is normal, so \bar{x} is exactly normal at every n.
- B. A is symmetric, so its means are close to normal at small n; B is strongly skewed, and its means’ skewness only falls like \displaystyle 1/\sqrt{n}.
- 0.2033: \mu_T = 1800, \displaystyle \sigma_T = 12\sqrt{100} = 120, \displaystyle z = \frac{100}{120} \approx 0.83, 1 - 0.7967 = 0.2033.
- From 332.40 to 567.60 dollars: \mu_T = 450, \displaystyle \sigma_T = 12\sqrt{25} = 60, 450 \pm 1.96(60) = 450 \pm 117.6.
- Total: \displaystyle 12\sqrt{9} = 36 dollars; mean: \displaystyle \frac{12}{\sqrt{9}} = 4 dollars. Variances add, so the total’s variance is 9 \times 144; dividing the total by 9 divides its standard deviation by 9.
- It says the distribution of \bar{x} approaches normal as n grows, with no threshold; 30 is a rule of thumb, too much for a symmetric population and too little for a very skewed one (Example 5).
- The orders are not independent draws from all orders: one family’s orders resemble each other, so they say little about other members, and the spread \displaystyle \sigma/\sqrt{n} and the normal shape are not promised.
- A sample’s histogram estimates the population’s shape, and more data show the skewed population more clearly. The sampling distribution of \bar{x} is a different object, the pattern of means over many samples, and the theorem says that pattern grows more normal.
- Nothing for the shape: \bar{x} is already exactly normal for every n. The mean \mu and spread \displaystyle \sigma/\sqrt{n} still hold.
- The mean’s skewness is the population’s divided by \displaystyle \sqrt{n}. Order totals: 1.33/6 \approx 0.22, nearly symmetric. Vouchers: \displaystyle 6.9/\sqrt{30} \approx 1.25, still strongly skewed, and the values are lumped on a few points.