Moment generating functions
Grace knows how each truckload reaching her granary is distributed. Adding the loads one convolution at a time is slow. Is there a single function of a distribution that turns sums into products, holds every moment at once, and still pins the distribution down?
Where you are 5 in Several random variables
Why this exists
Sums of independent variables finds the distribution of a sum of independent variables by convolution, one integral per term, with a separate computation for each family. Grace wants the distribution of a morning’s grain at her granary; Andre wants the time a batch spends in several dryers. Each further term is another convolution.
The exponential turns sums into products, e^{t(x + y)} = e^{tx} e^{ty}. Averaging e^{tX} over the distribution of X gives a function of t that keeps this property: for independent terms, the function of a sum is the product of the functions. It also stores every moment E(X^n) as a coefficient of its Maclaurin series, and when it is finite near t = 0 it determines the distribution.
Definition
In plain words: weight each value x of X by e^{tx} and take the mean. For t > 0 large values count for more; for t < 0 small ones do.
Let X be a random variable. Its moment generating function is
defined at every real t for which this mean is finite. By Result 1 of Expectation, linearity and indicators it is \sum_x e^{tx} p_X(x) for discrete X and \int_{-\infty}^{\infty} e^{tx} f_X(x) \, dx when X has a density. Since e^{tX} > 0, the mean is a positive number or +\infty, and M_X(0) = 1. It depends only on the distribution of X. We say X has a moment generating function when there is an h > 0 with M_X(t) finite for every t in (-h, h).
The set where M_X is finite is an interval containing 0. If t_1 < t < t_2, then e^{tx} \le e^{t_2 x} for x \ge 0 and e^{tx} \le e^{t_1 x} for x < 0, so e^{tx} \le e^{t_1 x} + e^{t_2 x}, and finiteness at t_1 and t_2 passes to t.
The nth moment of X is E(X^n), for n = 0, 1, 2, \dots, when E(|X|^n) is finite.
Intuition
Figure 1 shows three distributions with mean 1: the constant X = 1, with M_X(t) = e^t; \operatorname{Uniform}(0, 2); and \operatorname{Exponential}(1). Each curve passes through (0, 1) with slope E(X) = 1 (Result 1), so all three touch the line 1 + t there. They bend away at a rate set by M_X''(0) = E(X^2) = \operatorname{Var}(X) + 1: the constant, with variance 0, least; the uniform, with variance \tfrac13, next; the exponential, with variance 1, most.
For independent X and Y, e^{tX} and e^{tY} are independent, so the mean of their product is the product of the means: one multiplication in place of a convolution. The trip back to the distribution is Result 4.
Results
Result 1 (moments). Let M_X(t) be finite for every t in (-h, h), where h > 0. Then every moment E(X^n) is finite, and for every t in (-h, h)
So M_X has derivatives of every order on (-h, h), M_X^{(n)}(0) = E(X^n), E(X) = M_X'(0) and \operatorname{Var}(X) = M_X''(0) - M_X'(0)^2.
Result 2 (linear change). For constants a and b, M_{aX + b}(t) = e^{bt} M_X(at) at every t with M_X(at) finite.
Result 3 (independent sums). If X and Y are independent and M_X(t) and M_Y(t) are finite, then M_{X + Y}(t) = M_X(t) \, M_Y(t). If X_1, \dots, X_n are independent, M_{X_1 + \dots + X_n}(t) = M_{X_1}(t) \cdots M_{X_n}(t) wherever every factor is finite.
Result 4 (uniqueness). If there is an h > 0 with M_X(t) and M_Y(t) finite and equal for every t in (-h, h), then X and Y have the same distribution: F_X = F_Y.
Result 5 (the families). With q = 1 - p and 0 < p < 1:
| X | M_X(t) | finite for |
|---|---|---|
| \operatorname{Binomial}(n, p) | (q + pe^t)^n | all t |
| \operatorname{Geometric}(p) | \dfrac{pe^t}{1 - qe^t} | t < -\ln q |
| \operatorname{Poisson}(\lambda) | e^{\lambda(e^t - 1)} | all t |
| \operatorname{Uniform}(a, b) | \dfrac{e^{tb} - e^{ta}}{t(b - a)}, and 1 at t = 0 | all t |
| \operatorname{Exponential}(\lambda) | \dfrac{\lambda}{\lambda - t} | t < \lambda |
| \operatorname{Gamma}(n, \lambda) | \left(\dfrac{\lambda}{\lambda - t}\right)^n | t < \lambda |
| N(\mu, \sigma^2) | e^{\mu t + \sigma^2 t^2 / 2} | all t |
Result 6 (families closed under sums). Let X_1, \dots, X_n be independent. (a) If X_i \sim \operatorname{Binomial}(m_i, p), the sum is \operatorname{Binomial}(m_1 + \dots + m_n, p). (b) If X_i \sim \operatorname{Poisson}(\lambda_i), the sum is \operatorname{Poisson}(\lambda_1 + \dots + \lambda_n). (c) If X_i \sim \operatorname{Gamma}(k_i, \lambda), with one rate, the sum is \operatorname{Gamma}(k_1 + \dots + k_n, \lambda). (d) If X_i \sim N(\mu_i, \sigma_i^2), the sum is N(\mu_1 + \dots + \mu_n, \ \sigma_1^2 + \dots + \sigma_n^2).
Parts (a), (b), (d) and the exponential case of (c) are Result 3 of Sums of independent variables, proved there by convolution.
Result 7 (Chernoff bound). For every real a and every t > 0 with M_X(t) finite,
so P(X \ge a) is at most the smallest value of the right side over t > 0. For t < 0 the same right side bounds P(X \le a).
Why it is true
Result 1. Take 0 < s < h. For real x, e^{s|x|} \le e^{sx} + e^{-sx}, so E(e^{s|X|}) \le M_X(s) + M_X(-s) < \infty. One term of the series for e^{s|x|} is at most the whole, \displaystyle \frac{(s|x|)^n}{n!} \le e^{s|x|}, so E(|X|^n) \le n! \, s^{-n} E(e^{s|X|}) is finite.
Now fix t with 0 < |t| < h (at t = 0 both sides are 1). The series for the exponential gives \displaystyle e^{tX} = \sum_n \frac{(tX)^n}{n!}. Split each term into its positive and negative parts, u = u^+ - u^-: e^{tX} = A - B with \displaystyle A = \sum_n \frac{((tX)^n)^+}{n!} and \displaystyle B = \sum_n \frac{((tX)^n)^-}{n!}, series of nonnegative variables with A + B = e^{|t| \, |X|}. The mean of a series of nonnegative random variables is the sum of their means: for discrete X this is exchanging the order of a double series of nonnegative terms, for a density integrating a nonnegative series term by term (in general, the monotone convergence theorem). So E(A) and E(B) are at most E(e^{|t| \, |X|}) < \infty, and
A power series converging on (-h, h) may be differentiated term by term there (Power series, proved), and its nth derivative at 0 is n! times the coefficient of t^n, here E(X^n). The variance formula is Result 1 of Variance and covariance.
Result 2. e^{t(aX + b)} = e^{bt} e^{(at)X}, and a constant factor comes out of the mean (Result 2 of Expectation, linearity and indicators).
Result 3. At t = 0 both sides are 1. For t \ne 0, x \mapsto e^{tx} is continuous and strictly monotone, so \{a < e^{tX} \le b\} = \{X \in J\} and \{c < e^{tY} \le d\} = \{Y \in K\} for intervals J and K. Result 3 of Independence of events and variables gives P(X \in J, \ Y \in K) = P(X \in J) \, P(Y \in K), the rectangle test for e^{tX} and e^{tY}. So these are independent, with finite means M_X(t) and M_Y(t). By Result 5 of Variance and covariance, E(e^{tX} e^{tY}) = M_X(t) M_Y(t), and e^{tX} e^{tY} = e^{t(X + Y)}. For n terms, X_1 + \dots + X_{n-1} is independent of X_n (Result 5 of Independence of events and variables); induct on n.
Result 4 is stated without proof. It is the uniqueness theorem for the two-sided The Laplace transform, proved in a course on measure-theoretic probability through the inversion formula for characteristic functions. Exercise 15 proves it for variables with finitely many values.
Result 5. Binomial: by the binomial theorem, \sum_k \binom{n}{k} (pe^t)^k q^{n-k} = (q + pe^t)^n. Geometric: \sum_{k \ge 1} e^{tk} q^{k-1} p = pe^t \sum_{j \ge 0} (qe^t)^j, an Infinite geometric series, finite exactly when qe^t < 1. Poisson: \displaystyle \sum_k e^{tk} e^{-\lambda} \frac{\lambda^k}{k!} = e^{-\lambda} e^{\lambda e^t}. Uniform: \displaystyle \int_a^b \frac{e^{tx}}{b - a} \, dx. Exponential: \int_0^\infty \lambda e^{-(\lambda - t)x} \, dx is \displaystyle \frac{\lambda}{\lambda - t} for t < \lambda and infinite for t \ge \lambda. Gamma: a \operatorname{Gamma}(n, \lambda) variable is distributed as a sum of n independent \operatorname{Exponential}(\lambda) variables (Result 3(c) of Sums of independent variables); apply Result 3. Normal: for Z \sim N(0, 1), \displaystyle tz - \frac{z^2}{2} = \frac{t^2}{2} - \frac{(z - t)^2}{2}, and the N(t, 1) density has total 1, so M_Z(t) = e^{t^2/2}. For N(\mu, \sigma^2), substituting x = \mu + \sigma z in the integral gives M_X(t) = e^{\mu t} M_Z(\sigma t).
Result 6. By Results 3 and 5 the sum has moment generating function \prod_i (q + pe^t)^{m_i}, \prod_i e^{\lambda_i (e^t - 1)}, \displaystyle \prod_i \big(\frac{\lambda}{\lambda - t}\big)^{k_i} or \prod_i e^{\mu_i t + \sigma_i^2 t^2/2}: the function of Result 5 for the stated family, finite on an interval around 0. Result 4 identifies the distribution.
Result 7. For t > 0, g(x) = e^{tx} is positive and increasing, so Result 2 of Markov and Chebyshev inequalities gives \displaystyle P(X \ge a) \le \frac{E(e^{tX})}{e^{ta}}. For t < 0 apply this to -X with -t > 0 and level -a.
Worked examples
Bins before a jam
The number X of bins Andre fills before the first auger jam of a day has P(X = k) = (\tfrac12)^{k+1} for k = 0, 1, 2, \dots. Find M_X, where it is finite, E(X) and \operatorname{Var}(X).
SolutionBy definition, M_X(t) = \sum_{k \ge 0} e^{tk} (\tfrac12)^{k+1} = \tfrac12 \sum_{k \ge 0} (\tfrac{e^t}{2})^k, a geometric series. It converges exactly when \displaystyle \frac{e^t}{2} < 1, that is t < \ln 2, and then
This is finite on (-\ln 2, \ln 2), so Result 1 applies. Differentiating, \displaystyle M_X'(t) = \frac{e^t}{(2 - e^t)^2} and \displaystyle M_X''(t) = \frac{e^t}{(2 - e^t)^2} + \frac{2e^{2t}}{(2 - e^t)^3}, so E(X) = M_X'(0) = 1, M_X''(0) = 3 and \operatorname{Var}(X) = 3 - 1^2 = 2. Check: X + 1 is \operatorname{Geometric}(\tfrac12), with mean 2 and variance \displaystyle \frac{1/2}{1/4} = 2 (Variance and covariance, Result 7).
Every moment at once
The gap X, in minutes, between trucks at Grace’s scale is exponential with mean 12. Find E(X^n) for every n, and the third central moment E((X - 12)^3).
SolutionThe rate is \lambda = \tfrac{1}{12}, so by Result 5, for |t| < \tfrac{1}{12},
By Result 1 the coefficient of t^n is \displaystyle \frac{E(X^n)}{n!}, so E(X^n) = n! \, 12^n: E(X) = 12, E(X^2) = 288, E(X^3) = 10{,}368. Expanding the cube and using linearity, with \mu = 12,
As \operatorname{Var}(X) = 288 - 144 = 144, the ratio \displaystyle \frac{3{,}456}{1{,}728} = 2 of the third central moment to \sigma^3 measures the long right tail.
Two drying stages
A batch spends T_1 \sim \operatorname{Gamma}(2, \tfrac12) hours in Andre’s first dryer and T_2 \sim \operatorname{Gamma}(3, \tfrac12) hours in the second, independently. Find the distribution of T = T_1 + T_2, its mean and variance, and P(T > 12).
SolutionSince \displaystyle \frac{1/2}{1/2 - t} = \frac{1}{1 - 2t}, Result 5 gives M_{T_1}(t) = (1 - 2t)^{-2} and M_{T_2}(t) = (1 - 2t)^{-3} for t < \tfrac12. By Result 3, M_T(t) = (1 - 2t)^{-5}, the \operatorname{Gamma}(5, \tfrac12) function, so T \sim \operatorname{Gamma}(5, \tfrac12) by Result 4. Differentiating, M_T'(t) = 10(1 - 2t)^{-6} and M_T''(t) = 120(1 - 2t)^{-7}, so E(T) = 10 hours and \operatorname{Var}(T) = 120 - 100 = 20. The gamma tail formula of Result 3(c) of Sums of independent variables, with \lambda \cdot 12 = 6, gives
Above the tangent
Let X have a finite mean \mu, and let M_X(t) be finite. Prove that M_X(t) \ge e^{\mu t}, and that equality at some t \ne 0 forces P(X = \mu) = 1.
SolutionFirst, e^u \ge 1 + u for every real u, with equality only at u = 0: g(u) = e^u - 1 - u has g'(u) = e^u - 1, negative for u < 0 and positive for u > 0, so g is strictly smallest at 0, where g(0) = 0. Put u = t(X - \mu) and multiply by e^{\mu t} > 0:
By linearity the right side has mean e^{\mu t}(1 + t \cdot 0) = e^{\mu t}, and taking means keeps the inequality, so M_X(t) \ge e^{\mu t}.
If equality holds, W = e^{tX} - e^{\mu t}(1 + t(X - \mu)) is nonnegative with mean 0, so P(W = 0) = 1 by the lemma in the proof of Result 5 of Markov and Chebyshev inequalities. As W = 0 only where t(X - \mu) = 0, and t \ne 0, P(X = \mu) = 1.
A weekly tail
The number X of loads the granary rejects in a week is \operatorname{Poisson}(4). Bound P(X \ge 12) by Markov’s, Chebyshev’s and the Chernoff bound, and compare with the exact value.
SolutionMarkov’s inequality gives \displaystyle \frac{E(X)}{12} = \tfrac13. Chebyshev’s, with \operatorname{Var}(X) = 4, bounds it by P(|X - 4| \ge 8), at most \displaystyle \frac{4}{64} = 0.0625 (Markov and Chebyshev inequalities, Results 1 and 3). By Results 5 and 7, for t > 0, P(X \ge 12) \le e^{h(t)} with
Then h'(t) = 4e^t - 12 is zero at t = \ln 3 and h''(t) = 4e^t > 0, so this is the smallest value (Figure 2), h(\ln 3) = 8 - 12 \ln 3, and
The exact value, \displaystyle 1 - \sum_{k=0}^{11} e^{-4} \frac{4^k}{k!} \approx 0.000915, is about a sixth of the Chernoff bound, while Chebyshev’s bound is about 68 times too large.
A morning’s grain
On a harvest morning the number N of trucks reaching Grace’s granary is \operatorname{Poisson}(6), and the loads X_1, X_2, \dots, in tons, are exponential with mean 20, independent of each other and of N. Let S = X_1 + \dots + X_N, with S = 0 when N = 0. Find M_S, E(S) and \operatorname{Var}(S).
SolutionAt each outcome exactly one indicator \mathbf{1}_{\{N = n\}} is 1, so
The terms are nonnegative, so the mean of the series is the sum of the means, as in the proof of Result 1. In each term the two factors are functions of N and of X_1, \dots, X_n, hence independent (Result 5 of Independence of events and variables), so for t < \tfrac{1}{20} the term has mean P(N = n) M_X(t)^n by Result 5 of Variance and covariance and Result 3 here, with \displaystyle M_X(t) = \frac{1}{1 - 20t}. This is the tower rule of Conditional expectation written out. Summing the Poisson series,
With \displaystyle \psi(t) = \frac{120t}{1 - 20t}, \displaystyle \psi' = \frac{120}{(1 - 20t)^2} and \displaystyle \psi'' = \frac{4{,}800}{(1 - 20t)^3}. As M_S = e^\psi, M_S' = \psi' e^\psi and M_S'' = (\psi'' + \psi'^2) e^\psi, so E(S) = 120 tons, M_S''(0) = 4{,}800 + 14{,}400, and \operatorname{Var}(S) = 19{,}200 - 14{,}400 = 4{,}800. Result 5 of Conditional expectation agrees: it gives E(N)\operatorname{Var}(X) + \operatorname{Var}(N) E(X)^2, which is 6 \cdot 400 + 6 \cdot 400 = 4{,}800.
Boundaries
Not every variable has one. If a tail is heavier than every exponential, M_X(t) is infinite for one sign of t (Exercise 18), or for every t \ne 0, as for the density \displaystyle \frac{1}{\pi(1 + x^2)}. Results 1, 4 and 7 then say nothing; the characteristic function E(e^{itX}), finite for every variable, takes over in a later course.
Moments alone are not enough. For Z \sim N(0, 1), Y = e^Z has every moment, E(Y^n) = M_Z(n) = e^{n^2/2}, yet M_Y(t) is infinite for every t > 0, since e^{te^z} e^{-z^2/2} \to \infty as z \to \infty. Other distributions share all of these moments (stated without proof), so Result 4 cannot be replaced by a list of moments.
Result 3 needs independence. With Y = -X, M_{X + Y}(t) = 1, while M_X(t) M_{-X}(t) = M_X(t) M_X(-t), which is at least e^{\mu t} e^{-\mu t} = 1 by Example 4, with equality at t \ne 0 only for a constant.
A Chernoff bound helps only above the mean. By Example 4, for t > 0, e^{-ta} M_X(t) \ge e^{t(\mu - a)}, which is at least 1 when a \le \mu.
Where this goes next
The The central limit theorem, proved is proved with these functions: the moment generating function of a standardized sum of n independent copies tends to e^{t^2/2}, that of N(0, 1), and a continuity theorem stated there turns this into convergence of distribution functions. The The bivariate normal is described by the joint function E(e^{sX + tY}).
Exercises
Moments from the function
- Overnight, the number of full bins changes by X, which is -1, 0 or 2 with probabilities \tfrac14, \tfrac12, \tfrac14. Find M_X, and E(X) and \operatorname{Var}(X) from it.
- Let X have density 2x on (0, 1). Find M_X(t) for t \ne 0, and read E(X^n) off its Maclaurin series.
- Let Z \sim N(0, 1). Use the series of e^{t^2/2} to find E(Z^4), E(Z^6) and E(Z^{2k}), and explain why the odd moments are 0.
- Let T \sim \operatorname{Gamma}(3, 2). Find E(T), \operatorname{Var}(T) and E(T^3) from the series of M_T.
Reading the distribution
- A count X has M_X(t) = (\tfrac14 + \tfrac34 e^t)^8. Name its distribution and find P(X = 6).
- A variable Y has M_Y(t) = e^{3(e^{2t} - 1)}. Show that Y has the distribution of 2X with X \sim \operatorname{Poisson}(3), and find P(Y = 4), P(Y = 3) and \operatorname{Var}(Y).
- A load’s moisture reading X has M_X(t) = e^{3t + t^2}. Name its distribution and find P(X > 5).
- The wait X, in hours, of a truck at the scale has \displaystyle M_X(t) = \tfrac13 + \frac{2}{3(1 - t)} for t < 1. Describe its distribution, and find E(X) and P(X > 2).
Sums and random sums
- Each moisture probe Andre pushes into a bin finds a wet pocket with probability \tfrac14, independently. Let T be the number of probes up to and including the third wet pocket. Find M_T, E(T) and \operatorname{Var}(T).
- Trucks arriving in an hour number X \sim \operatorname{Poisson}(2) and trucks leaving Y \sim \operatorname{Poisson}(3), independently. Find M_D for D = X - Y, and E(D) and \operatorname{Var}(D) from it.
- A sampling spear lands X meters east and Y meters north of its target, independent and each N(0, 1). Find M_{X^2}, show that R^2 = X^2 + Y^2 is exponential, and find P(R > 2).
- Andre unloads trucks one after another, each taking an exponential time with mean 10 minutes, and stops after the first truck whose grain is wet; each truck is wet with probability \tfrac15, independently of everything else. Find the distribution of his total unloading time S and P(S > 60).
Tail bounds
- Each of 100 sacks fails inspection with probability 0.1, independently, and X is the number that fail. Find the Chernoff bound for P(X \ge 20), and compare it with Chebyshev’s bound and the exact value.
- Let Z \sim N(0, 1). Show that the Chernoff bound gives P(Z \ge a) \le e^{-a^2/2} for a > 0, and compare it with the exact value at a = 3.
Proofs
- Let X and Y take values in a finite set \{x_1, \dots, x_m\}, and let M_X = M_Y on an open interval around 0. Prove that X and Y have the same pmf.
- Let M_X(t) be finite and equal to M_X(-t) for every t in (-h, h), h > 0. Prove that -X has the distribution of X and that every odd moment of X is 0.
- Let X have a moment generating function, and K_X = \ln M_X. Prove that K_X'(0) = E(X) and K_X''(0) = \operatorname{Var}(X), and that K_{X+Y} = K_X + K_Y for independent X and Y that both have one.
- Let \displaystyle P(X = k) = \frac{c}{k^2} for k = 1, 2, 3, \dots, with \displaystyle c = \frac{6}{\pi^2}. Prove that M_X(t) is finite exactly for t \le 0, so X has no moment generating function.
- Let X \sim \operatorname{Poisson}(\lambda) and a > \lambda. Prove that \displaystyle P(X \ge a) \le e^{-\lambda} \left( \frac{e \lambda}{a} \right)^a.
- Let X \sim \operatorname{Binomial}(n, p) be independent of Y, which has a moment generating function, and let X + Y \sim \operatorname{Binomial}(n + m, p). Prove that Y \sim \operatorname{Binomial}(m, p).
Answers
- M_X(t) = \tfrac14 e^{-t} + \tfrac12 + \tfrac14 e^{2t} for all t; E(X) = \tfrac14 and \operatorname{Var}(X) = \tfrac{19}{16}. M_X'(0) = -\tfrac14 + \tfrac12 and M_X''(0) = \tfrac14 + 1 = \tfrac54, so the variance is \tfrac54 - \tfrac{1}{16}.
- \displaystyle M_X(t) = \frac{2(1 + (t - 1)e^t)}{t^2} and \displaystyle E(X^n) = \frac{2}{n + 2}. By parts, \int_0^1 2x e^{tx} \, dx is \displaystyle \frac{2e^t}{t} - \frac{2(e^t - 1)}{t^2}. The series of 1 + (t - 1)e^t is \displaystyle \sum_{k \ge 2} \frac{(k - 1) t^k}{k!}, so \displaystyle M_X(t) = \sum_{n \ge 0} \frac{2(n + 1)}{(n + 2)!} t^n, and by Result 1 \displaystyle E(X^n) = n! \cdot \frac{2(n + 1)}{(n + 2)!}.
- E(Z^4) = 3, E(Z^6) = 15 and \displaystyle E(Z^{2k}) = \frac{(2k)!}{2^k k!}. For all t, \displaystyle e^{t^2/2} = \sum_k \frac{t^{2k}}{2^k k!}; matching the coefficient of t^{2k} with \displaystyle \frac{E(Z^{2k})}{(2k)!} (Result 1) gives the even moments, and the series has no odd powers.
- E(T) = \tfrac32, \operatorname{Var}(T) = \tfrac34 and E(T^3) = \tfrac{15}{2}. \displaystyle M_T(t) = (1 - \tfrac{t}{2})^{-3} = \sum_n \binom{n + 2}{2} \frac{t^n}{2^n} (differentiate the geometric series twice), so E(T^n) = n! \binom{n + 2}{2} 2^{-n}, giving \tfrac32, 3, \tfrac{15}{2}; the variance is 3 - \tfrac94.
- \operatorname{Binomial}(8, \tfrac34) by Results 4 and 5, and P(X = 6) = \binom86 (\tfrac34)^6 (\tfrac14)^2 = \tfrac{5103}{16384} \approx 0.3115.
- By Result 2, M_{2X}(t) = M_X(2t) = e^{3(e^{2t} - 1)} for all t, so Result 4 gives the claim. Then P(Y = 4) = P(X = 2) = \tfrac92 e^{-3} \approx 0.2240, P(Y = 3) = 0 as 2X is even, and \operatorname{Var}(Y) = 4 \cdot 3 = 12.
- N(3, 2), as \mu = 3 and \displaystyle \frac{\sigma^2}{2} = 1; \displaystyle P(X > 5) = 1 - \Phi(\frac{2}{\sqrt2}) = 1 - \Phi(\sqrt2) \approx 0.0786.
- No wait with probability \tfrac13, otherwise an \operatorname{Exponential}(1) wait: E(X) = \tfrac23 and P(X > 2) = \tfrac23 e^{-2} \approx 0.0902. Let B be an indicator with P(B = 1) = \tfrac23, independent of V \sim \operatorname{Exponential}(1), and W = BV. Splitting on B, E(e^{tW}) = \tfrac13 + \tfrac23 E(e^{tV}), which is M_X(t); Result 4 gives X the distribution of W. M_X'(0) = \tfrac23.
- \displaystyle M_T(t) = \left(\frac{e^t}{4 - 3e^t}\right)^3 for t < \ln \tfrac43, E(T) = 12 and \operatorname{Var}(T) = 36. T is a sum of three independent \operatorname{Geometric}(\tfrac14) variables; use Results 3 and 5. With Exercise 17, K_T(t) = 3t - 3\ln(4 - 3e^t), \displaystyle K_T'(t) = 3 + \frac{9e^t}{4 - 3e^t} and \displaystyle K_T''(t) = \frac{36e^t}{(4 - 3e^t)^2}, which are 12 and 36 at 0.
- M_D(t) = e^{2(e^t - 1) + 3(e^{-t} - 1)} for all t, E(D) = -1 and \operatorname{Var}(D) = 5. By Result 2, M_{-Y}(t) = M_Y(-t), and -Y is independent of X, so Result 3 applies. With Exercise 17, K_D' = 2e^t - 3e^{-t} and K_D'' = 2e^t + 3e^{-t} at 0.
- M_{X^2}(t) = (1 - 2t)^{-1/2} for t < \tfrac12, R^2 \sim \operatorname{Exponential}(\tfrac12), and P(R > 2) = e^{-2} \approx 0.1353. The integrand \displaystyle e^{tx^2} \frac{e^{-x^2/2}}{\sqrt{2\pi}} equals \displaystyle \frac{e^{-(1 - 2t)x^2/2}}{\sqrt{2\pi}}, which is (1 - 2t)^{-1/2} times the \displaystyle N(0, \frac{1}{1 - 2t}) density for t < \tfrac12, and does not decay for t \ge \tfrac12. By Result 3, \displaystyle M_{R^2}(t) = \frac{1}{1 - 2t} = \frac{1/2}{1/2 - t}; Result 4 names it, and P(R^2 > 4) = e^{-4/2}.
- S \sim \operatorname{Exponential}(\tfrac{1}{50}) and P(S > 60) = e^{-6/5} \approx 0.3012. The number N of trucks is \operatorname{Geometric}(\tfrac15). As in Example 6, with \displaystyle M = \frac{\lambda}{\lambda - t}, \lambda = \tfrac{1}{10}, p = \tfrac15 and q = \tfrac45, M_S(t) = \sum_{n \ge 1} q^{n-1} p M^n, which is \displaystyle \frac{pM}{1 - qM} = \frac{p\lambda}{p\lambda - t} for t < p\lambda = \tfrac{1}{50}. Result 4 finishes.
- Chernoff: (\tfrac49)^{20} (\tfrac98)^{100} \approx 0.0118; Chebyshev: \tfrac{9}{100}; exact \approx 0.0020. For t > 0 the bound is e^{h(t)} with h(t) = -20t + 100 \ln(0.9 + 0.1e^t); h'(t) = 0 gives 10e^t = 18 + 2e^t, so e^t = \tfrac94 and 0.9 + 0.1e^t = \tfrac98. Chebyshev: \displaystyle P(X \ge 20) \le P(|X - 10| \ge 10) \le \frac{9}{100}.
- e^{-ta} e^{t^2/2} is smallest at t = a, with value e^{-a^2/2}. At a = 3 the bound is e^{-9/2} \approx 0.0111 and the exact value 1 - \Phi(3) \approx 0.0013.
- Let c_k = P(X = x_k) - P(Y = x_k) with the x_k distinct. Then \sum_k c_k e^{t x_k} = M_X(t) - M_Y(t) = 0 on an interval around 0, so every derivative is 0 at t = 0: \sum_k c_k x_k^n = 0 for n = 0, 1, \dots, m - 1. The matrix with entries x_k^n is a Vandermonde matrix, with determinant \prod_{j < k} (x_k - x_j) \ne 0, so every c_k = 0.
- By Result 2 with a = -1, M_{-X}(t) = M_X(-t) = M_X(t) on (-h, h), so Result 4 gives -X the distribution of X. M_X is even; the derivative of an even function is odd and of an odd function even, so every odd-order derivative is odd and vanishes at 0. By Result 1, E(X^n) = M_X^{(n)}(0) = 0 for odd n.
- M_X > 0, so K_X is defined on (-h, h), with \displaystyle K_X' = \frac{M_X'}{M_X} and \displaystyle K_X'' = \frac{M_X'' M_X - (M_X')^2}{M_X^2}. At 0, M_X = 1, so Result 1 gives K_X'(0) = E(X) and K_X''(0) = E(X^2) - E(X)^2. For independent X and Y, Result 3 gives M_{X+Y} = M_X M_Y near 0, and the logarithm of a product is the sum of the logarithms.
- For t \le 0 the terms \displaystyle e^{tk} \frac{c}{k^2} are at most \displaystyle \frac{c}{k^2}, whose sum is 1. For t > 0, \displaystyle e^{tk} \ge \frac{(tk)^3}{6}, so the terms are at least \displaystyle \frac{ct^3 k}{6} and do not tend to 0; by the The nth term test for divergence the series diverges and M_X(t) = \infty. No interval (-h, h) has M_X finite.
- By Results 5 and 7, P(X \ge a) \le e^{g(t)} for t > 0, with g(t) = \lambda(e^t - 1) - at. g'(t) = \lambda e^t - a is zero at \displaystyle t = \ln \frac{a}{\lambda}, positive as a > \lambda. There \displaystyle g = a - \lambda - a \ln \frac{a}{\lambda}, and \displaystyle e^g = e^{-\lambda} e^a (\frac{\lambda}{a})^a, the stated bound.
- Let M_Y be finite on (-h, h). There Result 3 gives (q + pe^t)^{n + m} = (q + pe^t)^n M_Y(t), and q + pe^t > 0, so M_Y(t) = (q + pe^t)^m, the \operatorname{Binomial}(m, p) function of Result 5. Result 4 finishes.