Discrete distributions: binomial, geometric, Poisson
Fix the number of tries and count successes, count tries until the first success, or count rare events in a fixed span of time. What are the distributions of these three counts, and how are they related?
Where you are 5 in Random variables
Why this exists
Probability axioms and sample spaces builds a probability P on a sample space \Omega. Most questions are about a number read off the outcome, such as how many of the morning’s chats are password resets. Such a number is a random variable, and what matters about it is how its probability is spread over its values, its distribution.
Three distributions come from one picture, a run of independent trials with a common chance of success. Fix the number of trials and count successes: binomial. Count trials until the first success: geometric. Let the number of trials grow while the chance of each shrinks, holding the expected count fixed: Poisson. The school lessons The binomial distribution and The geometric distribution met the first two; here all three are proved, with the limit that joins the first to the third.
Definition
A random variable is a function X : \Omega \to \mathbb{R} such that \{X \le x\} is an event for every real x; when \Omega is finite or countable this holds automatically. The school version is Random variables. X is discrete when there is a finite or countably infinite set S of reals with P(X \in S) = 1.
The probability mass function (pmf) of a discrete X is p_X(x) = P(X = x), and the values with p_X(x) > 0 form its support. The distribution function of any random variable is F_X(x) = P(X \le x) for real x. X \sim (a named distribution) means that X has that pmf.
The indicator of an event A is \mathbf{1}_A(\omega) = 1 when \omega \in A and 0 otherwise, so P(\mathbf{1}_A = 1) = P(A).
Trials are independent when the probability of any combination of their results is the product of the separate probabilities, as in Independent events and the multiplication rule; the general definition is in Independence of events and variables. The three families:
Binomial. X \sim \operatorname{Binomial}(n, p), for a whole number n \ge 1 and 0 \le p \le 1, when for k = 0, 1, \dots, n
It is the distribution of the number of successes in n independent trials, each a success with probability p.
Geometric. X \sim \operatorname{Geometric}(p), for 0 < p \le 1, when p_X(k) = (1 - p)^{k - 1} p for k = 1, 2, 3, \dots. It is the distribution of the number of trials up to and including the first success, in independent trials each a success with probability p.
Poisson. X \sim \operatorname{Poisson}(\lambda), for \lambda > 0, when for k = 0, 1, 2, \dots
It models counts of rare, independent events in a fixed span (Result 5).
Intuition
A pmf places the unit of probability mass at a list of points, drawn as bars. The distribution function is their running total from the left: a staircase climbing by p_X(k) at each value (Figure 2, in Example 1).
The Poisson family is the binomial seen from far away. Split an hour into n short slots, each holding an email with probability \lambda / n, independently. The count is \operatorname{Binomial}(n, \lambda / n), with mean \lambda whatever n is. As the slots shrink, the count stops depending on n and its pmf settles on e^{-\lambda} \lambda^k / k!. Figure 1 shows \lambda = 2 with only n = 10; Example 5 takes n = 400.
Results
Result 1 (mass functions). Let X be discrete with support S. Then p_X(x) \ge 0, \sum_{x \in S} p_X(x) = 1, and P(X \in B) = \sum_{x \in B \cap S} p_X(x) for every set B of reals for which \{X \in B\} is an event. Conversely, if S is finite or countably infinite and p : S \to [0, \infty) has \sum_{x \in S} p(x) = 1, then some random variable has pmf p.
Result 2 (distribution functions). For any random variable X, F_X is nondecreasing and continuous from the right, with F_X(x) \to 0 as x \to -\infty and F_X(x) \to 1 as x \to \infty. For a < b, P(a < X \le b) = F_X(b) - F_X(a). At every x, P(X = x) = F_X(x) - F_X(x^-), where F_X(x^-) is the limit of F_X(y) as y increases to x: the jump equals the mass.
Result 3 (the three families). Each of the three formulas in the Definition is a pmf. The number of successes in n independent trials with success probability p is \operatorname{Binomial}(n, p), and the number of trials up to and including the first success is \operatorname{Geometric}(p).
Result 4 (geometric tail and lack of memory). If X \sim \operatorname{Geometric}(p) with p < 1, then P(X > k) = (1 - p)^k for k = 0, 1, 2, \dots, and for all such m and k,
Example 6 proves the converse: no other distribution on \{1, 2, 3, \dots\} has this property.
Result 5 (Poisson limit). Let \lambda > 0, and let p_n \in (0, 1) with n p_n \to \lambda as n \to \infty. Then for every fixed k = 0, 1, 2, \dots,
Result 6 (means). The mean of a discrete X is E(X) = \sum_{x \in S} x \, p_X(x), provided \sum_{x \in S} |x| \, p_X(x) converges (the weighted mean of Expected value; the full theory is Expectation, linearity and indicators). A \operatorname{Binomial}(n, p) variable has mean np, as in Mean and standard deviation of a binomial; a \operatorname{Geometric}(p) variable has mean 1/p, as in Mean and standard deviation of a geometric random variable; a \operatorname{Poisson}(\lambda) variable has mean \lambda. Variances are in Variance and covariance.
Why it is true
Result 1. Each p_X(x) is a probability, so it is at least 0. The events \{X = x\}, x \in S, are disjoint and countably many with union \{X \in S\}, so countable additivity gives \sum_{x \in S} p_X(x) = 1. Likewise \{X \in B\} is the disjoint union of the events \{X = x\}, x \in B \cap S, and a part of \{X \notin S\}, which has probability 0. For the converse, take \Omega = S with P(A) = \sum_{\omega \in A} p(\omega), a probability by Result 7 of Probability axioms and sample spaces, and X(\omega) = \omega.
Result 2. If x < y then \{X \le x\} \subseteq \{X \le y\}, so F_X is nondecreasing. The rest is continuity, Result 8 of Probability axioms and sample spaces. The events \{X \le x + 1/n\} shrink to \{X \le x\}, so F_X(x + 1/n) \to F_X(x), and because F_X is nondecreasing the limit from the right along every sequence is the same. The events \{X \le n\} grow to \Omega and \{X \le -n\} shrink to \varnothing, since every X(\omega) is a real number. \{X \le b\} is the disjoint union of \{X \le a\} and \{a < X \le b\}, which gives the difference formula. Finally \{x - 1/n < X \le x\} shrinks to \{X = x\}, so
Result 3, binomial. Record n trials as a string of S (success) and F (failure). By independence a string with k successes has probability p^k (1 - p)^{n - k}, and \{X = k\} is the disjoint union of the \binom{n}{k} such strings, one for each choice of positions for the S’s (Counting models). The binomial theorem gives the total (p + (1 - p))^n = 1.
Result 3, geometric. \{X = k\} is the event that the first k - 1 trials fail and trial k succeeds, of probability (1 - p)^{k - 1} p. Since 0 \le 1 - p < 1, the geometric series gives the total p / (1 - (1 - p)) = 1. The outcomes in which every trial fails give X no value; they lie inside “the first m trials fail” for every m, so their probability is at most (1 - p)^m \to 0.
Result 3, Poisson. The terms are positive, and the series for e^x gives the total e^{-\lambda} e^{\lambda} = 1.
Result 4. Summing a geometric tail,
Since \{X > m + k\} \subseteq \{X > m\}, the Finding a conditional probability gives P(X > m + k \mid X > m) = (1 - p)^{m + k} / (1 - p)^m, which is (1 - p)^k.
Result 5. Fix k and split the binomial probability into four factors:
The first is a product of k factors (n - j)/n, each tending to 1. The second tends to \lambda^k / k!. Since n p_n \to \lambda, p_n \to 0, so the fourth tends to 1. For the third, n \ln(1 - p_n) = n p_n \cdot \ln(1 - p_n)/p_n, and \ln(1 - u)/u \to -1 as u \to 0 by l’Hôpital’s rule, so n \ln(1 - p_n) \to -\lambda and, e^x being continuous, (1 - p_n)^n \to e^{-\lambda}.
Result 6. The values are nonnegative, so convergence is absolute. Binomial: k \binom{n}{k} = n \binom{n - 1}{k - 1}, so with j = k - 1,
the sum being the \operatorname{Binomial}(n - 1, p) total. Geometric: differentiating \sum_{k \ge 0} q^k = 1/(1 - q) term by term for |q| < 1, as Power series, proved allows, gives \sum_{k \ge 1} k q^{k - 1} = 1/(1 - q)^2; with q = 1 - p, E(X) = p / p^2 = 1/p. Poisson: the k = 0 term vanishes, and with j = k - 1,
Worked examples
Escalations in a shift
The number X of tickets Ben escalates to the network team in a shift has p_X(k) = c(4 - k) for k = 0, 1, 2, 3, and 0 otherwise. Find c, write out F_X, and use it to find P(1 \le X \le 2) and P(X > 1.5).
SolutionBy Result 1 the masses total 1: 10c = 1, so c = \tfrac{1}{10} and the masses at 0, 1, 2, 3 are 0.4, 0.3, 0.2, 0.1. F_X(x) adds the masses at values up to x:
Each step is closed at its left end (Figure 2): F_X(1) = 0.7, because \{X \le 1\} includes X = 1. This is the continuity from the right of Result 2. Since X takes only whole-number values, \{1 \le X \le 2\} = \{0 < X \le 2\}, and Result 2 gives P(1 \le X \le 2) = F_X(2) - F_X(0) = 0.9 - 0.4 = 0.5, agreeing with 0.3 + 0.2. Also P(X > 1.5) = 1 - F_X(1.5) = 1 - 0.7 = 0.3.
Password resets
Each of the next 12 chats Aisha answers is a password reset with probability \tfrac{1}{4}, independently. Let X be the number of resets. Find P(X = 3), P(X \ge 2), and the most likely value of X.
SolutionA fixed number of independent trials with a common success probability (The binomial setting) makes X \sim \operatorname{Binomial}(12, \tfrac{1}{4}) by Result 3. So
“At least two” is the complement of “zero or one”:
For the most likely value, compare neighbors. Since \binom{12}{k+1} / \binom{12}{k} = (12 - k)/(k + 1),
This exceeds 1 exactly when 12 - k > 3k + 3, that is, k < \tfrac{9}{4}. The masses rise up to k = 3 and fall after it, so X = 3 is the most likely value. Here it equals the mean np = 3.
Restarting a kiosk
A self-service kiosk at the helpdesk has frozen. Each restart clears the fault with probability 0.35, independently of earlier restarts. Let X be the number of restarts up to and including the one that clears it. Find P(X = 2), P(X \le 3) and E(X). Given that 3 restarts have already failed, find the probability that more than 2 further restarts are needed.
SolutionBy Result 3, X \sim \operatorname{Geometric}(0.35). The first restart fails and the second succeeds, so P(X = 2) = 0.65 \cdot 0.35 = 0.2275. By Result 4, P(X \le 3) = 1 - 0.65^3 = 1 - 0.274625 = 0.725375, and by Result 6, E(X) = 1/0.35 = \tfrac{20}{7} \approx 2.86.
Three failures is the event X > 3, and more than two further restarts is X > 5. Result 4 gives
The failures change nothing; a fault that grew harder to clear with each failure would not be geometric.
Overnight email
Model the number X of emails reaching the helpdesk between 2 and 3 a.m. as \operatorname{Poisson}(1.2). Find P(X = 0), P(X = 2) and P(X \ge 3). For what \lambda would an hour with no email have probability exactly \tfrac{1}{2}?
SolutionResult 5 fits: many customers, each with a small independent chance of writing in that hour, and 1.2 is the mean count (Result 6). From the pmf,
The complement of X \ge 3 is X \le 2, so
An hour with no email has probability e^{-\lambda}, which equals \tfrac{1}{2} exactly when \lambda = \ln 2 \approx 0.693.
Damaged loaners
Aisha’s desk lends out 400 laptops in a term, and each comes back damaged with probability 0.005, independently. Compare the exact probability that at most one comes back damaged with its Poisson approximation.
SolutionThe number damaged is X \sim \operatorname{Binomial}(400, 0.005). Exactly,
Here np = 2 and p is small, so Result 5 suggests \operatorname{Poisson}(2):
The two differ by less than 0.001, and single masses agree as closely: P(X = 2) is 0.27135 exactly against 0.27067. The Poisson formula depends on n and p only through np.
Only the geometric forgets
Let X take values in \{1, 2, 3, \dots\} with P(X > m + k) = P(X > m) \, P(X > k) for all whole numbers m, k \ge 0. (When P(X > m) > 0 this says P(X > m + k \mid X > m) = P(X > k).) Prove that X \sim \operatorname{Geometric}(p) with p = P(X = 1) > 0.
SolutionLet G(k) = P(X > k) and q = G(1). Since X \ge 1, G(0) = 1. The hypothesis with m = 1 gives G(k + 1) = q \, G(k), and induction gives G(k) = q^k for every k \ge 0.
Next, q < 1. The events \{X > k\} shrink as k grows and have empty intersection, since every X(\omega) is a number, so continuity (Result 8 of Probability axioms and sample spaces) gives G(k) \to 0, impossible if q = 1.
Now \{X > k - 1\} is the disjoint union of \{X = k\} and \{X > k\}, so for k \ge 1
the \operatorname{Geometric}(1 - q) pmf, and 1 - q = P(X = 1) = p is positive because q < 1.
Boundaries
The binomial needs its whole setting: a number of trials fixed in advance, independent trials, and the same p for each. Drawing without replacement from a finite pile breaks independence; the count is then hypergeometric, treated in Counting models.
Two conventions for the geometric. Some books count the failures before the first success, X - 1, with pmf (1 - p)^k p for k = 0, 1, 2, \dots and mean (1 - p)/p. With p = 0 success never comes and there is no geometric distribution.
The Poisson is a model and an approximation. It gives positive probability to every whole number, so a count that cannot exceed n is only approximated by it. It needs events that occur independently, one at a time, at a steady rate; events in bursts, such as tickets after an outage, spread out more than a Poisson count with the same mean.
Result 5 is a limit for fixed k. It gives no error bound for a particular n, and with p not small the fit is rough, as Figure 1 shows at k = 0 and k = 2.
A pmf describes discrete variables only. For a variable with a density, P(X = x) = 0 for every x; the distribution function still works, and Densities and cumulative distribution functions builds on it.
Where this goes next
Expectation, linearity and indicators writes a binomial count as a sum of n indicators, one per trial, and gets E(X) = np with no binomial coefficients. In Densities and cumulative distribution functions the distribution function carries over unchanged, and the exponential distribution appears as the continuous partner of the geometric, a waiting time without memory. For large np(1 - p), the Shape of a binomial and the normal approximation approximates the binomial instead.
Exercises
Mass functions and distribution functions
- A discrete random variable has p_X(k) = c k^2 for k = 1, 2, 3, 4, and 0 otherwise. Find c, P(X \ge 3) and F_X(2.5).
- A random variable X has F_X(x) = 0 for x < 0, 0.25 for 0 \le x < 2, 0.6 for 2 \le x < 5, and 1 for x \ge 5. Find the pmf of X, P(X > 1) and P(2 \le X < 5).
- A random variable has p_X(k) = c \cdot 4^k / k! for k = 0, 1, 2, \dots. Find c, name the distribution, and find P(X \le 1).
Binomial
- Each of the 15 tickets Aisha opens today is closed within an hour with probability 0.8, independently. Find the probability that exactly 12 are closed within an hour, and that at least 14 are.
- Each ticket in Ben’s queue is a duplicate of an earlier one with probability 0.1, independently. What is the smallest number of tickets he must read for the chance of finding at least one duplicate to be at least 0.9?
- Let X \sim \operatorname{Binomial}(20, 0.35). Find the most likely value of X by comparing p_X(k + 1) with p_X(k).
Geometric
- Let X \sim \operatorname{Geometric}(0.2). Find P(X = 4), P(X > 5) and P(3 \le X \le 6).
- X is geometric and P(X > 2) = 0.49. Find p and P(X = 2).
- Show that for X \sim \operatorname{Geometric}(p), F_X(x) = 0 for x < 1 and F_X(x) = 1 - (1 - p)^{\lfloor x \rfloor} for x \ge 1. Then, for p = 0.25, find the smallest whole number m with F_X(m) \ge \tfrac{1}{2}.
Poisson
- The number of jams in the helpdesk printer in a day is \operatorname{Poisson}(0.8). Find the probability of no jams, of exactly 2, and of at least 2.
- Each of 1{,}000 user accounts is locked out on a given day with probability 0.003, independently. Find the probability of no lockouts and of at most 2, exactly and by the Poisson approximation.
Choosing a model
- Name the distribution and its parameters, or say why none of the three fits. (a) The number of printer tickets among the next 30, each about printers with probability 0.15, independently. (b) The number of tickets Ben opens until the first printer ticket, with the same assumptions. (c) The number of printer tickets in a day, when they arrive independently, one at a time, at an average rate of 4 a day. (d) The number of printer tickets among 5 drawn without replacement from a pile of 20, of which 6 are about printers.
Proofs
- Let X \sim \operatorname{Poisson}(\lambda). Prove that the most likely value of X is \lfloor \lambda \rfloor when \lambda is not a whole number, and that \lambda - 1 and \lambda tie for most likely when it is.
- Let X take values in \{0, 1, 2, \dots\} and have a finite mean. Prove that E(X) = \sum_{k=0}^{\infty} P(X > k), and use it to find the mean of a \operatorname{Geometric}(p) variable without differentiating a series.
- The number N of calls to the helpdesk in a morning is \operatorname{Poisson}(\lambda). Each call concerns hardware with probability p > 0, independently of the other calls and of N, so that given N = n the number H of hardware calls is \operatorname{Binomial}(n, p). Prove that H \sim \operatorname{Poisson}(\lambda p). With \lambda = 10 and p = 0.25, find the probability of a morning with no hardware calls.
- Let X \sim \operatorname{Poisson}(\lambda) and Y \sim \operatorname{Poisson}(\mu) be independent: P(X = i, Y = j) = P(X = i) P(Y = j) for all i, j. Prove that X + Y \sim \operatorname{Poisson}(\lambda + \mu).
Answers
- c = \tfrac{1}{30}, P(X \ge 3) = \tfrac{5}{6}, F_X(2.5) = \tfrac{1}{6}. The total is c(1 + 4 + 9 + 16) = 30c = 1; then P(X \ge 3) = \tfrac{9 + 16}{30} and F_X(2.5) = P(X \le 2) = \tfrac{1 + 4}{30}.
- p_X(0) = 0.25, p_X(2) = 0.35, p_X(5) = 0.4; P(X > 1) = 0.75; P(2 \le X < 5) = 0.35. By Result 2 the masses are the jumps. Then P(X > 1) = 1 - F_X(1), and the only value in [2, 5) is 2.
- c = e^{-4}, \operatorname{Poisson}(4), P(X \le 1) = 5e^{-4} \approx 0.0916. Since \sum_k 4^k / k! = e^4, the total is 1 exactly when c = e^{-4}; then P(X \le 1) = e^{-4}(1 + 4).
- \approx 0.2501 and \approx 0.1671. X \sim \operatorname{Binomial}(15, 0.8): P(X = 12) = 455 \cdot 0.8^{12} \cdot 0.2^3, and P(X \ge 14) = 15 \cdot 0.8^{14} \cdot 0.2 + 0.8^{15} = 3.8 \cdot 0.8^{14}.
- 22 tickets. With n tickets, 1 - 0.9^n \ge 0.9 means 0.9^n \le 0.1, that is, n \ge \ln 0.1 / \ln 0.9 \approx 21.85. Check: 1 - 0.9^{21} \approx 0.8906 and 1 - 0.9^{22} \approx 0.9015.
- 7, with p_X(7) \approx 0.1844. The ratio \displaystyle \frac{p_X(k+1)}{p_X(k)} = \frac{20 - k}{k + 1} \cdot \frac{0.35}{0.65} = \frac{7(20 - k)}{13(k + 1)} exceeds 1 exactly when 140 - 7k > 13k + 13, that is, k < 6.35, so the masses rise up to k = 7 and fall after it.
- P(X = 4) = 0.8^3 \cdot 0.2 = 0.1024; P(X > 5) = 0.8^5 = 0.32768; and P(3 \le X \le 6) = P(X > 2) - P(X > 6), which is 0.64 - 0.262144 = 0.377856.
- p = 0.3 and P(X = 2) = 0.21. By Result 4, (1 - p)^2 = 0.49, so 1 - p = 0.7; then P(X = 2) = 0.7 \cdot 0.3.
- m = 3. X takes whole-number values from 1 on, so F_X(x) = 0 for x < 1, and for x \ge 1, F_X(x) = P(X \le \lfloor x \rfloor) = 1 - (1 - p)^{\lfloor x \rfloor} by Result 4. With p = 0.25 we need 0.75^m \le 0.5, that is, m \ge \ln 0.5 / \ln 0.75 \approx 2.41: indeed F_X(2) = 0.4375 and F_X(3) = 0.578125.
- e^{-0.8} \approx 0.4493; 0.32 e^{-0.8} \approx 0.1438; 1 - 1.8 e^{-0.8} \approx 0.1912. The middle one is \tfrac{0.8^2}{2} e^{-0.8}, and the last is 1 - e^{-0.8}(1 + 0.8).
- Exactly, 0.997^{1000} \approx 0.04956 and P(X \le 2) \approx 0.42285, summing \binom{1000}{k} 0.003^k \, 0.997^{1000 - k} for k \le 2. With \lambda = 1000 \cdot 0.003 = 3: e^{-3} \approx 0.04979 and e^{-3}(1 + 3 + 4.5) = 8.5 e^{-3} \approx 0.42319.
- (a) \operatorname{Binomial}(30, 0.15): fixed number of independent trials, constant p. (b) \operatorname{Geometric}(0.15): trials up to and including the first success. (c) \operatorname{Poisson}(4): independent events one at a time at a steady rate, with mean 4. (d) None: draws without replacement are not independent, so the count is hypergeometric, as in Counting models.
- For k \ge 0, p_X(k + 1) / p_X(k) = \lambda / (k + 1), so p_X(k + 1) is larger than, equal to or smaller than p_X(k) as k + 1 is less than, equal to or greater than \lambda. If \lambda is not a whole number, the masses strictly rise up to k = \lfloor \lambda \rfloor and strictly fall after it. If it is, they rise up to \lambda - 1, repeat that value at \lambda, and then fall.
- P(X > k) = \sum_{j > k} p_X(j), so \sum_{k \ge 0} P(X > k) = \sum_{k \ge 0} \sum_{j > k} p_X(j). The terms are nonnegative, so the order of summation may be exchanged (the fact about nonnegative series used in Result 7 of Probability axioms and sample spaces). A fixed p_X(j) appears once for each k = 0, \dots, j - 1, that is, j times, so the sum is \sum_{j \ge 1} j \, p_X(j) = E(X). For \operatorname{Geometric}(p), Result 4 gives \displaystyle E(X) = \sum_{k \ge 0} (1 - p)^k = \frac{1}{p}.
- The events \{N = n\} are disjoint with total probability 1, so by the The law of total probability, for k \ge 0, \displaystyle P(H = k) = \sum_{n \ge k} e^{-\lambda} \frac{\lambda^n}{n!} \binom{n}{k} p^k (1 - p)^{n - k}. Using \displaystyle \frac{1}{n!} \binom{n}{k} = \frac{1}{k! \, (n - k)!}, \lambda^n = \lambda^k \lambda^{n - k} and j = n - k, this is \displaystyle e^{-\lambda} \frac{(\lambda p)^k}{k!} \sum_{j \ge 0} \frac{(\lambda (1 - p))^j}{j!}. The sum is e^{\lambda (1 - p)}, leaving \displaystyle e^{-\lambda p} \frac{(\lambda p)^k}{k!}. With \lambda = 10 and p = 0.25, H \sim \operatorname{Poisson}(2.5) and P(H = 0) = e^{-2.5} \approx 0.0821.
- For n \ge 0, \{X + Y = n\} is the disjoint union of the events \{X = i, Y = n - i\}, i = 0, \dots, n. By independence each has probability \displaystyle e^{-\lambda} \frac{\lambda^i}{i!} \cdot e^{-\mu} \frac{\mu^{n - i}}{(n - i)!} = \frac{e^{-(\lambda + \mu)}}{n!} \binom{n}{i} \lambda^i \mu^{n - i}. Summing over i, the binomial theorem gives \displaystyle P(X + Y = n) = e^{-(\lambda + \mu)} \frac{(\lambda + \mu)^n}{n!}, the \operatorname{Poisson}(\lambda + \mu) pmf.