The central limit theorem, proved
Each of Sarah’s beehives yields an unpredictable amount of honey, from a distribution nobody has drawn. Why does the total from many hives follow the normal curve all the same, and what does it take to prove it?
Where you are 4 in Limit theorems
Why this exists
The law of large numbers says that the average \bar{X}_n of n independent copies of X settles on \mu = E(X), and \displaystyle \operatorname{Var}(\bar{X}_n) = \frac{\sigma^2}{n} says the error is typically of size \displaystyle \frac{\sigma}{\sqrt{n}}. Neither says how the error is distributed. Sarah keeps 64 beehives and wants the chance that the season’s total honey passes a target. Chebyshev’s inequality (Markov and Chebyshev inequalities) gives only a crude bound, and the exact distribution of the total is a 64-fold convolution (Sums of independent variables) of a yield distribution she does not know.
The school lesson The central limit theorem states the answer and uses it: a standardized sum is approximately normal, whatever the distribution of one term. This lesson proves it. Moment generating functions turn the sum into a product, the product into the limit e^{t^2/2}, and a theorem linking moment generating functions to distribution functions carries the limit back to probabilities.
Definition
In plain words: a sequence of random variables converges in distribution when the probabilities P(Y_n \le x) approach those of a limit variable. Nothing is said about the values of Y_n at any one outcome.
Let Y_1, Y_2, \dots and Y be random variables with distribution functions F_{Y_n} and F_Y. The sequence converges in distribution to Y, written Y_n \xrightarrow{d} Y, if
at every x where F_Y is continuous. The exception is needed for limits with jumps: \displaystyle Y_n = \frac1n ought to converge to the constant 0, yet F_{Y_n}(0) = 0 for every n while the constant has distribution function 1 at 0.
The standard normal distribution function is
the function behind the standard normal table. It is continuous everywhere, so Y_n \xrightarrow{d} N(0, 1) means F_{Y_n}(x) \to \Phi(x) at every real x.
Let X_1, X_2, \dots be independent and identically distributed (iid) with mean \mu and variance \sigma^2, where 0 < \sigma^2 < \infty, and let S_n = X_1 + \dots + X_n. Since E(S_n) = n\mu and \operatorname{Var}(S_n) = n\sigma^2, the standardized sum is
with E(Z_n) = 0 and \operatorname{Var}(Z_n) = 1 for every n.
Intuition
Figure 1 takes X_i \sim \operatorname{Exponential}(1), a strongly right-skewed term with \mu = \sigma = 1. Then S_n \sim \operatorname{Gamma}(n, 1) (Result 6(c) of Moment generating functions), and the density of Z_n is \displaystyle \sqrt{n} f_{S_n}(n + \sqrt{n} z) by Functions of a random variable. At n = 1 it is the exponential density moved to start at -1; at n = 4 the skew is still plain; at n = 16 the curve nearly lies on the dashed normal density.
Write \displaystyle Y = \frac{X - \mu}{\sigma} for one standardized term, so \displaystyle Z_n = \frac{Y_1 + \dots + Y_n}{\sqrt{n}}. By Results 2 and 3 of Moment generating functions,
Since E(Y) = 0 and E(Y^2) = 1, the series of Result 1 there begins \displaystyle M_Y(s) = 1 + \frac{s^2}{2} + \frac{E(Y^3)}{6}s^3 + \cdots. Put \displaystyle s = \frac{t}{\sqrt{n}} and raise to the nth power: the s^2 term gives the factor e^{t^2/2} whatever Y is, while the s^3 term carries an extra \displaystyle \frac{1}{\sqrt{n}} and later terms smaller factors still. Every feature of the term distribution beyond its mean and variance is washed out, and what remains, e^{t^2/2}, is the moment generating function of N(0, 1). Figure 2 shows this for the terms of Figure 1.
Results
Result 1 (continuity theorem, stated without proof). Let Y_1, Y_2, \dots and Y be random variables, and let there be an h > 0 such that M_Y(t) and every M_{Y_n}(t) are finite for |t| < h, with
for every t in (-h, h). Then Y_n \xrightarrow{d} Y.
Result 2 (central limit theorem). Let X_1, X_2, \dots be iid with a moment generating function (finite on some interval (-h, h)), mean \mu and variance \sigma^2 > 0. Then Z_n \xrightarrow{d} N(0, 1): for every real x,
Result 3 (intervals and uniformity). Let Y_n have distribution function F_n, with F_n(x) \to \Phi(x) at every x. (a) For every x, P(Y_n < x) \to \Phi(x) and P(Y_n = x) \to 0; so for a \le b, each of P(a < Y_n \le b), P(a \le Y_n \le b) and P(a < Y_n < b) tends to \Phi(b) - \Phi(a). (b) The convergence is uniform: \sup_x |F_n(x) - \Phi(x)| \to 0.
Result 4 (finite variance, stated without proof). The conclusion of Result 2 holds for every iid sequence with 0 < \sigma^2 < \infty, with no assumption on the moment generating function. This is the Lindeberg-Lévy theorem.
Result 5 (binomial). Let S_n \sim \operatorname{Binomial}(n, p) with 0 < p < 1 and q = 1 - p. Then \displaystyle \frac{S_n - np}{\sqrt{npq}} \xrightarrow{d} N(0, 1). This is the de Moivre-Laplace theorem, behind the school lesson Shape of a binomial and the normal approximation.
In use, these limits are read as approximations at one large n. Boundaries says what is known about the error.
Why it is true
Result 1 is not proved here; it is due to Curtiss (1942). In a course on measure-theoretic probability it follows from Lévy’s continuity theorem for characteristic functions E(e^{itX}), and the uniqueness of that transform.
Result 2. One standardized term. Let \displaystyle Y_i = \frac{X_i - \mu}{\sigma}. These are iid (Result 5 of Independence of events and variables), with E(Y_i) = 0 by linearity and \displaystyle \operatorname{Var}(Y_i) = \frac{\sigma^2}{\sigma^2} = 1. By Result 2 of Moment generating functions, \displaystyle M_Y(s) = e^{-\mu s/\sigma} M_X(\frac{s}{\sigma}) is finite for |s| < h\sigma; put h' = h\sigma.
The product. \displaystyle Z_n = \frac{1}{\sqrt{n}}(Y_1 + \dots + Y_n), so Results 3 and 2 there give \displaystyle M_{Z_n}(t) = M_Y\big(\frac{t}{\sqrt{n}}\big)^n, finite for |t| < h' and every n.
The expansion. By Result 1 there, with E(Y) = 0 and E(Y^2) = 1, for |s| < h',
For s \ne 0, \displaystyle \frac{r(s)}{s^2} = \sum_{k \ge 3} \frac{E(Y^k)}{k!} s^{k-2}, a power series converging on (-h', h'). A power series is continuous inside its interval of convergence (Power series, proved), and this one is 0 at s = 0, so \displaystyle \frac{r(s)}{s^2} \to 0 as s \to 0.
Fix t \ne 0 in (-h', h'); at t = 0 both sides of the limit below are 1. With \displaystyle s_n = \frac{t}{\sqrt{n}} \to 0,
A lemma: if a_n \to a, then \displaystyle (1 + \frac{a_n}{n})^n \to e^a. For \displaystyle |u| \le \frac12, the series \displaystyle \ln(1 + u) = u - \frac{u^2}{2} + \frac{u^3}{3} - \cdots gives
The a_n are bounded, so \displaystyle |\frac{a_n}{n}| \le \frac12 for large n, and then \displaystyle n \ln(1 + \frac{a_n}{n}) = a_n + \rho_n with \displaystyle |\rho_n| \le n \cdot \frac{a_n^2}{n^2} = \frac{a_n^2}{n} \to 0. So \displaystyle n \ln(1 + \frac{a_n}{n}) \to a, and since the exponential is continuous, \displaystyle (1 + \frac{a_n}{n})^n \to e^a.
Conclusion. M_{Z_n}(t) \to e^{t^2/2} for every t in (-h', h'), and e^{t^2/2} = M_Z(t) for Z \sim N(0, 1) by Result 5 of Moment generating functions. Result 1 gives Z_n \xrightarrow{d} Z, and since \Phi is continuous, P(Z_n \le x) \to \Phi(x) at every x.
Result 3. (a) Fix x and \varepsilon > 0, and by continuity of \Phi choose \delta > 0 with \Phi(x) - \Phi(x - \delta) < \varepsilon. Since \{Y_n \le x - \delta\} \subseteq \{Y_n < x\} \subseteq \{Y_n \le x\},
For large n, F_n(x - \delta) > \Phi(x - \delta) - \varepsilon > \Phi(x) - 2\varepsilon and F_n(x) < \Phi(x) + \varepsilon, so |P(Y_n < x) - \Phi(x)| < 2\varepsilon. Then P(Y_n = x) = F_n(x) - P(Y_n < x) \to 0, and each interval probability is a difference of two of these limits; for example P(a \le Y_n \le b) = F_n(b) - P(Y_n < a).
(b) Fix \varepsilon > 0. As \Phi is continuous and increases from 0 to 1, there are points x_1 < \dots < x_{k-1} with \Phi(x_1) \le \varepsilon, 1 - \Phi(x_{k-1}) \le \varepsilon and \Phi(x_j) - \Phi(x_{j-1}) \le \varepsilon. Put x_0 = -\infty and x_k = \infty, where every distribution function is 0 and 1. For n beyond some N, |F_n(x_j) - \Phi(x_j)| < \varepsilon at the finitely many x_j. For x between x_{j-1} and x_j, both functions are nondecreasing, so
So \sup_x |F_n(x) - \Phi(x)| \le 2\varepsilon for n > N.
Result 4 is not proved here. Its proof runs the argument above with characteristic functions, which exist for every random variable, and a second-order Taylor bound in place of the power series; it is Theorem 27.1 of Billingsley’s Probability and Measure.
Result 5. By Result 6(a) of Moment generating functions with every m_i = 1, S_n has the distribution of a sum of n independent indicators, each with mean p, variance pq (Result 7 of Variance and covariance with one trial) and moment generating function q + pe^t, finite for every t. Result 2 applies to that sum, and the statement concerns only the distribution of S_n.
Worked examples
A season’s honey
Sarah’s 64 hives yield amounts of honey, in pounds, that are iid with mean 48 and standard deviation 12, and no hive yields more than 120 pounds. Approximate the probability that the total passes 3200 pounds, and the total that is exceeded with probability about 0.95.
SolutionA yield lies between 0 and 120, so e^{tX_1} \le e^{120|t|} and M_{X_1}(t) is finite for every t: Result 2 applies. The total S has mean 64 \cdot 48 = 3072 and standard deviation \displaystyle 12\sqrt{64} = 96. By Results 2 and 3, read at n = 64,
For the second part we want s with \displaystyle \Phi\big(\frac{s - 3072}{96}\big) \approx 0.05. Since \Phi(-1.6449) \approx 0.05, s \approx 3072 - 1.6449(96) \approx 2914.1 pounds. Only the mean, standard deviation and boundedness were used.
Mites in two hundred hives
Each of Ethan’s 200 hives carries varroa mites with probability 0.3, independently. Approximate the probability that at most 50 carry mites, with and without the half-unit correction, and compare with the exact value.
SolutionThe count S \sim \operatorname{Binomial}(200, 0.3) has mean 60 and variance 200(0.3)(0.7) = 42, so \displaystyle \sqrt{42} \approx 6.4807, and Result 5 applies. Read directly,
But S takes only integer values, so P(S \le 50) = P(S \le c) for every c in [50, 51), while the normal values at these c differ. The midpoint 50.5, the continuity correction of Shape of a binomial and the normal approximation, splits the difference:
Summing the binomial pmf over k = 0, \dots, 50 gives the exact value 0.0695. The corrected value is off by about 0.002, the uncorrected by 0.008.
Twenty-five foraging trips
David times a forager bee’s trips. The trip times are iid exponential with mean 20 minutes. Approximate the probability that 25 trips take more than 600 minutes in total, and compare with the exact value.
SolutionOne trip time has \mu = \sigma = 20 and moment generating function \displaystyle \frac{1}{1 - 20t} for \displaystyle t < \frac{1}{20} (Result 5 of Moment generating functions), finite near 0. The total T has mean 500 and standard deviation \displaystyle 20\sqrt{25} = 100, so
Exactly, \displaystyle T \sim \operatorname{Gamma}(25, \frac{1}{20}) by Result 6(c) of Moment generating functions, and the gamma tail formula of Result 3(c) of Sums of independent variables, with \displaystyle \frac{600}{20} = 30, gives
How many hives
With the yields of Example 1 (mean 48 pounds, standard deviation 12), how many hives must Sarah keep for the total to reach 5000 pounds with probability about 0.99?
SolutionWith n hives the total S_n has mean 48n and standard deviation \displaystyle 12\sqrt{n}. By Result 2, \displaystyle P(S_n \ge 5000) \approx 1 - \Phi\big(\frac{5000 - 48n}{12\sqrt{n}}\big). Since \Phi(-2.3263) \approx 0.01 and \Phi is increasing, this is at least 0.99 when \displaystyle \frac{5000 - 48n}{12\sqrt{n}} \le -2.3263. Multiply by \displaystyle 12\sqrt{n} > 0, use 2.3263(12) \approx 27.916, and put \displaystyle u = \sqrt{n}:
The quadratic opens upward with one positive root,
so the condition holds for u \ge 10.501, that is n \ge 10.501^2 \approx 110.27. Sarah needs 111 hives. Check: \displaystyle 48n - 27.916\sqrt{n} is about 5033.9 at n = 111 and about 4987.2 at n = 110.
A Poisson limit
Prove that
Let X_1, X_2, \dots be iid \operatorname{Poisson}(1). By Result 6(b) of Moment generating functions, S_n \sim \operatorname{Poisson}(n), so the expression is exactly P(S_n \le n). Each X_i has \mu = \sigma^2 = 1 and moment generating function e^{e^t - 1}, finite for every t (Result 5 there), so Result 2 applies. The event S_n \le n is the event Z_n \le 0, hence
The approach is slow: at n = 10 the expression is 0.5830.
Stings in a season
Let N_\lambda \sim \operatorname{Poisson}(\lambda) and \displaystyle W_\lambda = \frac{N_\lambda - \lambda}{\sqrt{\lambda}}. Prove that W_{\lambda_k} \xrightarrow{d} N(0, 1) for every sequence of real numbers \lambda_k \to \infty, integer or not. Then approximate the probability that David, stung a \operatorname{Poisson}(49) number of times in a season, is stung at least 60 times.
SolutionN_\lambda is a sum of iid terms only for integer \lambda, so we go to Result 1 directly. By Results 2 and 5 of Moment generating functions, for every real t, with \displaystyle s = \frac{t}{\sqrt{\lambda}},
Taylor’s theorem with the Lagrange remainder gives \displaystyle e^s = 1 + s + \frac{s^2}{2} + \frac{e^{\xi}}{6}s^3 for some \xi between 0 and s, so \displaystyle e^s - 1 - s = \frac{s^2}{2} + \rho with \displaystyle |\rho| \le \frac{|s|^3 e^{|s|}}{6}. As \lambda s^2 = t^2,
Along \lambda_k \to \infty the bound tends to 0, so M_{W_{\lambda_k}}(t) \to e^{t^2/2} for every t, and Result 1 gives W_{\lambda_k} \xrightarrow{d} N(0, 1).
For David, \lambda = 49 and \displaystyle \sqrt{\lambda} = 7. The count is an integer, so with the half-unit correction of Example 2,
The exact value, \displaystyle 1 - e^{-49}\sum_{k=0}^{59} \frac{49^k}{k!}, is 0.0703.
Boundaries
A finite variance is needed. For iid Cauchy terms, \bar{X}_n has the Cauchy distribution for every n (Boundaries of The law of large numbers), so no scaling by \displaystyle \sqrt{n} gives a normal limit. Terms with a finite mean and infinite variance, such as the density \displaystyle \frac{3}{2}x^{-5/2} on x > 1, need a scaling faster than \displaystyle \sqrt{n} and have non-normal limits, the stable laws; we state this without proof.
The moment generating function is the proof’s hypothesis, not the theorem’s. The density 3x^{-4} on x > 1 has E(X^2) = 3, finite, but M_X(t) = \infty for every t > 0. Result 4 covers it; Result 2 does not.
Distribution functions, not densities or point masses. Result 2 says nothing about P(S_n = k) or about the density of Z_n. The point probabilities of an integer-valued sum tend to 0 (Exercise 18); approximating them by the normal density is the work of local limit theorems, which need further hypotheses.
No rate comes with the theorem, and the tails are the weak spot. The Berry-Esseen theorem, stated without proof, bounds \sup_x |F_{Z_n}(x) - \Phi(x)| by \displaystyle C \, \frac{E(|X - \mu|^3)}{\sigma^3\sqrt{n}} with an absolute constant C; Shevtsova (2011) showed that C = 0.4748 works. That is an absolute error, and far in a tail it can exceed the probability itself. For 16 terms of \operatorname{Exponential}(1), P(Z_{16} > 3) = P(S_{16} > 28), which is \displaystyle e^{-28}\sum_{k=0}^{15}\frac{28^k}{k!} \approx 0.0054, about four times 1 - \Phi(3) \approx 0.0013. For far tails the Chernoff bound, Result 7 of Moment generating functions, is the better tool.
Independence and a common distribution. If X_1 = X_2 = \cdots, then S_n = nX_1 has standard deviation n\sigma, and \displaystyle \frac{S_n - n\mu}{n\sigma} = \frac{X_1 - \mu}{\sigma} for every n, normal only if X_1 is. Independent terms with different distributions still give a normal limit under Lindeberg’s condition, which in effect forbids any single term from carrying a fixed share of the variance (stated without proof; Theorem 27.2 of Billingsley).
Where this goes next
For random vectors with finite covariances, the same proof run with the moment generating function of a vector gives a multivariate central limit theorem; its two-dimensional limit is the The bivariate normal distribution. In Markov chains the terms are dependent, yet time averages along a chain obey a central limit theorem of their own. In statistics the theorem justifies the interval \displaystyle \bar{x} \pm 1.96\frac{s}{\sqrt{n}} for a mean: s converges in probability to \sigma (Example 4 and Result 5 of The law of large numbers), and Exercise 17 shows that rescalings tending to constants keep the normal limit. The estimate of about 62 trips in Example 5 of Markov and Chebyshev inequalities is this theorem read the way Example 4 reads it.
Exercises
Sums and averages
- Ethan’s honey jars hold weights, in ounces, that are iid with mean 16.2 and standard deviation 0.4, each between 15 and 18. Approximate the probability that a case of 49 jars weighs less than 790 ounces.
- David records 300 hive weights, each rounded to the nearest pound; the rounding errors are iid \displaystyle \operatorname{Uniform}(-\frac12, \frac12). Let E be their total. Approximate P(|E| \le 5), and find c with P(|E| \le c) \approx 0.99.
- The number K of brood frames in one of Sarah’s hives has P(K = 8) = 0.2, P(K = 9) = 0.5 and P(K = 10) = 0.3, independently across hives. Approximate, with the half-unit correction, the probability that 120 hives hold at least 1100 brood frames in total.
- The mite count in a sample from one hive has mean 2.5, standard deviation 1.5 and a moment generating function, independently across hives. Approximate the probability that the average count over 80 hives exceeds 2.8.
Binomial, Poisson and exponential totals
- Sarah introduces 400 new queens, each accepted with probability 0.8, independently. Approximate, with the half-unit correction, the probability that at least 330 are accepted, and compare with the exact value 0.1164.
- The number N of swarm calls Ethan answers in a season is \operatorname{Poisson}(25). Approximate P(N \le 20) with the half-unit correction, and write the exact value as a sum.
- Hive inspections take iid exponential times with mean 15 minutes. Approximate the probability that 36 inspections take at most 480 minutes in total, and find the exact value from the gamma tail formula.
How large a sample
- Honey per hive has standard deviation 6 pounds and a moment generating function. How many hives must David weigh for the average to lie within 1 pound of the mean with probability about 0.95? How many does Chebyshev’s inequality need to guarantee it?
- How many hives must be checked for the share carrying mites to be estimated within 0.03 with probability about 0.95, whatever the true share p? Use \displaystyle p(1 - p) \le \frac14.
- A frame yields honey with mean 5 pounds and standard deviation 2, independently, with a moment generating function. How many frames does Ethan need for the total to reach 400 pounds with probability about 0.95?
Limits through the theorem
- Let the X_i be iid with \displaystyle P(X_i = 1) = P(X_i = -1) = \frac12, a scout bee’s right or left turn at each fork. Show that \displaystyle M_{Z_n}(t) = \cosh\big(\frac{t}{\sqrt{n}}\big)^n, evaluate it at n = 4, t = 1, and prove directly that it tends to e^{t^2/2}.
- For iid \operatorname{Exponential}(1) terms, show that \displaystyle M_{Z_n}(t) = e^{-t\sqrt{n}}\big(1 - \frac{t}{\sqrt{n}}\big)^{-n} for \displaystyle t < \sqrt{n}, and evaluate it at t = 1 for n = 4, 16 and 100.
- Find \displaystyle \displaystyle\lim_{n \to \infty} \int_0^{n + \sqrt{n}} \frac{x^{n-1} e^{-x}}{(n-1)!} \, dx.
- Find \displaystyle \displaystyle\lim_{n \to \infty} 2^{-n} \sum_{0 \le k \le n/2 + \sqrt{n}} \binom{n}{k}.
Proofs
- Under the hypotheses of Result 2, prove from Results 2 and 3, without Chebyshev’s inequality, that P(|\bar{X}_n - \mu| \ge \varepsilon) \to 0 for every \varepsilon > 0.
- Let c be a constant and Y_n \xrightarrow{d} c. Prove that P(|Y_n - c| \ge \varepsilon) \to 0 for every \varepsilon > 0.
- Let Y_n \xrightarrow{d} N(0, 1), and let a_n \to a > 0 and b_n \to b be constants. Prove that \displaystyle P(a_n Y_n + b_n \le x) \to \Phi\big(\frac{x - b}{a}\big) for every x.
- Under the hypotheses of Result 2, prove that \sup_y P(S_n = y) \to 0.
- Refute the claim that whenever Y_n \xrightarrow{d} Y and Y_n' \xrightarrow{d} Y, also Y_n - Y_n' \xrightarrow{d} 0.
Answers
- The weights are bounded, so the moment generating function is finite everywhere. The total has mean 49(16.2) = 793.8 and standard deviation \displaystyle 0.4\sqrt{49} = 2.8, so P(S < 790) is about \displaystyle \Phi\big(\frac{-3.8}{2.8}\big) \approx \Phi(-1.357) \approx 0.0874.
- Each error is bounded, with mean 0 and variance \displaystyle \frac{1}{12}, so \displaystyle \operatorname{Var}(E) = \frac{300}{12} = 25. Then P(|E| \le 5) = P(|Z_{300}| \le 1), about 2\Phi(1) - 1 \approx 0.6827. Since \Phi(2.5758) \approx 0.995, c \approx 2.5758(5) \approx 12.88.
- E(K) = 9.1, E(K^2) = 83.3 and \operatorname{Var}(K) = 83.3 - 82.81 = 0.49. The total has mean 1092, variance 58.8 and standard deviation \approx 7.668. Being an integer, it is at least 1100 exactly when it exceeds 1099.5, with probability about \displaystyle 1 - \Phi\big(\frac{7.5}{7.668}\big), that is 1 - \Phi(0.978) \approx 0.164.
- \displaystyle P(\bar{X}_{80} > 2.8) = P\big(Z_{80} > \frac{0.3\sqrt{80}}{1.5}\big), and \displaystyle \frac{0.3\sqrt{80}}{1.5} \approx 1.789, so the probability is \approx 1 - \Phi(1.789) \approx 0.0368.
- Mean 320, variance 400(0.8)(0.2) = 64, standard deviation 8. By Result 5, P(S \ge 329.5) \approx 1 - \Phi(1.1875) \approx 0.1175, within 0.0011 of the exact 0.1164.
- By Example 6 with \lambda = 25, P(N \le 20.5) is about \displaystyle \Phi\big(\frac{-4.5}{5}\big) = \Phi(-0.9) \approx 0.1841. The exact value is \displaystyle e^{-25}\sum_{k=0}^{20} \frac{25^k}{k!} \approx 0.1855.
- The total T has mean 540 and standard deviation \displaystyle 15\sqrt{36} = 90, so \displaystyle P(T \le 480) \approx \Phi\big(-\frac23\big) \approx 0.2525. Exactly, \displaystyle T \sim \operatorname{Gamma}(36, \frac{1}{15}) and \displaystyle \frac{480}{15} = 32, so P(T \le 480) is \displaystyle 1 - e^{-32}\sum_{k=0}^{35} \frac{32^k}{k!} \approx 0.2621.
- \displaystyle P(|\bar{X}_n - \mu| \le 1) \approx 2\Phi\big(\frac{\sqrt{n}}{6}\big) - 1 \ge 0.95 when \displaystyle \frac{\sqrt{n}}{6} \ge 1.96, so n \ge (1.96 \cdot 6)^2 \approx 138.3: 139 hives. Chebyshev’s bound \displaystyle \frac{36}{n} is at most 0.05 only when n \ge 720.
- The share has standard deviation \displaystyle \sqrt{\frac{p(1 - p)}{n}} \le \frac{1}{2\sqrt{n}}, so it suffices that \displaystyle 0.03 \cdot 2\sqrt{n} \ge 1.96: \displaystyle n \ge \big(\frac{1.96}{0.06}\big)^2 \approx 1067.1, so 1068 hives.
- Need \displaystyle \frac{400 - 5n}{2\sqrt{n}} \le -1.6449, that is 5u^2 - 3.2898u - 400 \ge 0 with \displaystyle u = \sqrt{n} and 1.6449(2) = 3.2898. The positive root is \displaystyle u = \frac{3.2898 + \sqrt{3.2898^2 + 8000}}{10} \approx 9.279, so n \ge 86.1: 87 frames. Check: \displaystyle 5n - 3.2898\sqrt{n} is about 399.5 at n = 86 and 404.3 at n = 87.
- Here \mu = 0, \sigma = 1 and M_X(s) = \cosh s, so Results 2 and 3 of Moment generating functions give \displaystyle M_{Z_n}(t) = \cosh\big(\frac{t}{\sqrt{n}}\big)^n. At n = 4, t = 1: \cosh(0.5)^4 \approx 1.6168, against e^{1/2} \approx 1.6487. Directly: \displaystyle \cosh s = 1 + \frac{s^2}{2} + r(s) with \displaystyle r(s) = \sum_{k \ge 2} \frac{s^{2k}}{(2k)!}, and (2k)! \ge 24(2k - 4)! gives \displaystyle 0 \le r(s) \le \frac{s^4}{24}\cosh s. So \displaystyle \cosh\big(\frac{t}{\sqrt{n}}\big) = 1 + \frac{a_n}{n} with \displaystyle a_n = \frac{t^2}{2} + nr\big(\frac{t}{\sqrt{n}}\big) \to \frac{t^2}{2}, as \displaystyle 0 \le nr\big(\frac{t}{\sqrt{n}}\big) \le \frac{t^4 \cosh t}{24n}; the lemma in the proof of Result 2 gives the limit.
- \displaystyle M_X(t) = \frac{1}{1 - t} for t < 1, and Y = X - 1 has \displaystyle M_Y(s) = \frac{e^{-s}}{1 - s} by Result 2 there; \displaystyle M_{Z_n}(t) = M_Y\big(\frac{t}{\sqrt{n}}\big)^n is the given formula. At t = 1: 16e^{-2} \approx 2.1654 for n = 4, \approx 1.8274 for n = 16 and \approx 1.7092 for n = 100, closing slowly on e^{1/2}.
- The integrand is the \operatorname{Gamma}(n, 1) density, the density of a sum S_n of n iid \operatorname{Exponential}(1) terms, with \mu = \sigma = 1. The integral is \displaystyle P(S_n \le n + \sqrt{n}) = P(Z_n \le 1), which tends to \Phi(1) \approx 0.8413.
- The sum is \displaystyle P(S_n \le \frac{n}{2} + \sqrt{n}) for \displaystyle S_n \sim \operatorname{Binomial}(n, \frac12), with mean \displaystyle \frac{n}{2} and standard deviation \displaystyle \frac{\sqrt{n}}{2}. The event is Z_n \le 2, so by Result 5 the limit is \Phi(2) \approx 0.9772.
- Fix \varepsilon > 0 and K > 0. The event |\bar{X}_n - \mu| \ge \varepsilon is \displaystyle |Z_n| \ge \frac{\varepsilon\sqrt{n}}{\sigma}, which for \displaystyle n \ge (\frac{K\sigma}{\varepsilon})^2 lies inside \{|Z_n| \ge K\}. By Result 3(a), P(|Z_n| \ge K) = 1 - P(-K < Z_n < K), which tends to 2(1 - \Phi(K)). Given \eta > 0, choose K with 2(1 - \Phi(K)) < \eta; then P(|\bar{X}_n - \mu| \ge \varepsilon) < 2\eta for all large n, so it tends to 0.
- The constant c has distribution function 0 for x < c and 1 for x \ge c, continuous except at c; so F_{Y_n}(x) \to 0 for x < c and \to 1 for x > c. Since |Y_n - c| \ge \varepsilon forces Y_n \le c - \varepsilon or \displaystyle Y_n > c + \frac{\varepsilon}{2}, the union bound makes P(|Y_n - c| \ge \varepsilon) at most \displaystyle F_{Y_n}(c - \varepsilon) + 1 - F_{Y_n}(c + \frac{\varepsilon}{2}), which tends to 0 + 1 - 1 = 0.
- For large n, a_n > 0, so P(a_n Y_n + b_n \le x) = F_n(x_n) with \displaystyle x_n = \frac{x - b_n}{a_n} \to \frac{x - b}{a} = x^*. Then |F_n(x_n) - \Phi(x^*)| is at most \sup_y |F_n(y) - \Phi(y)| + |\Phi(x_n) - \Phi(x^*)|; the first term tends to 0 by Result 3(b), the second by continuity of \Phi.
- S_n = y exactly when Z_n = z with \displaystyle z = \frac{y - n\mu}{\sigma\sqrt{n}}. Let D_n = \sup_x |F_{Z_n}(x) - \Phi(x)|, which tends to 0 by Results 2 and 3(b). For any z and \delta > 0, P(Z_n = z) \le F_{Z_n}(z) - F_{Z_n}(z - \delta), which is at most \Phi(z) - \Phi(z - \delta) + 2D_n and so at most \displaystyle \frac{\delta}{\sqrt{2\pi}} + 2D_n, since the normal density is at most \displaystyle \frac{1}{\sqrt{2\pi}}. The bound does not depend on z, so \displaystyle \sup_y P(S_n = y) < \frac{2\delta}{\sqrt{2\pi}} for large n, and \delta is arbitrary.
- Let Z \sim N(0, 1). By symmetry of the density, P(-Z \le x) = 1 - \Phi(-x) = \Phi(x), so -Z \sim N(0, 1) too. Take Y_n = Z and Y_n' = -Z for every n: both converge in distribution to N(0, 1). But Y_n - Y_n' = 2Z, and at the continuity point \displaystyle x = \frac12 of the constant 0, \displaystyle P(2Z \le \frac12) = \Phi(\frac14) \approx 0.5987 for every n, not 1.