Sampling distribution of a difference
At one farmers market more shoppers pay cash than at the other. If a random sample is taken at each, how far apart will the two sample shares land, and how often will they rank the markets the wrong way around?
Where you are 7 in Sampling distributions
Why this exists
Riverside and Hilltop, two neighboring towns, each hold a Saturday farmers market. The two markets, their shoppers and every number in this lesson are made up. The market association wants to compare them: at which market do more shoppers pay cash, and at which do shoppers spend more? Pema, Dolma and Lhakpa run its surveys, and each survey takes one random sample at each market.
The answer to such a question is a gap between two sample statistics, such as the Riverside sample share minus the Hilltop sample share. Draw new samples and the gap changes. To know whether an observed gap reflects a real difference or only the luck of two draws, we need the pattern the gap makes over all the pairs of samples that could have been drawn. That pattern is the sampling distribution of a difference.
Two earlier lessons supply the pieces. In Sampling distribution of a sample proportion, the share \hat{p} of composting households in a sample centered at p with standard deviation \displaystyle \sqrt{p(1-p)/n}, close to normal under the large-counts condition. In Sampling distribution of a sample mean, the mean weight \bar{x} of a sample of coffee bags centered at \mu with standard deviation \displaystyle \sigma/\sqrt{n}, normal when the population is normal or n is large. This lesson puts two such statistics side by side. Day 1 is a difference of two proportions, Day 2 a difference of two means.
Definition
Take a random sample of size n_1 from population 1 and a random sample of size n_2 from population 2, drawn independently: which members land in one sample has no bearing on which land in the other. The sampling distribution of \hat{p}_1 - \hat{p}_2 is the distribution of the values that difference takes over all possible pairs of samples of these sizes. The sampling distribution of \bar{x}_1 - \bar{x}_2 is defined the same way for sample means. Their means and standard deviations are written \mu_{\hat{p}_1 - \hat{p}_2}, \sigma_{\hat{p}_1 - \hat{p}_2}, \mu_{\bar{x}_1 - \bar{x}_2} and \sigma_{\bar{x}_1 - \bar{x}_2}.
Throughout, population 1 is Riverside and population 2 is Hilltop, so a positive difference means Riverside’s statistic is larger. The association’s records for this season:
| Riverside (1) | Hilltop (2) | |
|---|---|---|
| shoppers this season | 5{,}000 | 3{,}200 |
| share who pay cash | p_1 = 0.55 | p_2 = 0.40 |
| mean spending per shopper | \mu_1 = 32 dollars | \mu_2 = 27 dollars |
| standard deviation of spending | \sigma_1 = 14 dollars | \sigma_2 = 11 dollars |
Intuition
Each sample share misses its own p by chance. The gap \hat{p}_1 - \hat{p}_2 collects both misses. When Riverside’s sample runs high and Hilltop’s runs low, the gap is too wide; when the reverse happens, it is too narrow. Because the samples are independent, neither miss is tied to the other, so they never cancel in a dependable way, and the gap varies more than either share alone. Figure 1 draws the two shares and their difference to one scale.
The gap does not vary as much as the two spreads added, though. That would need both misses to go the widest way on every draw, and independent misses seldom do. The exact rule is that the variances add, as in Combining random variables, even though the means subtract.
Results
Day 1: a difference of two proportions
For independent random samples of sizes n_1 and n_2:
The center holds for any two random samples. The spread needs:
- Independence between the groups: two independent random samples, or two groups formed by random assignment in a Completely randomized design and replication.
- Independence within each sample: the The ten percent condition for each sample drawn without replacement, n_1 \le 0.10N_1 and n_2 \le 0.10N_2. It is not needed for groups formed by random assignment, since nothing is sampled from a larger population.
The shape is approximately normal under the large-counts condition for each group: n_1p_1, n_1(1-p_1), n_2p_2 and n_2(1-p_2) all at least 10. Then chances come from
and the areas of the The standard normal distribution.
Day 2: a difference of two means
For independent random samples of sizes n_1 and n_2:
with the same two independence conditions as Day 1. The shape is normal under the normal condition for each group: in each group separately, the population is normal (any sample size), or the sample is large enough for the The central limit theorem (a common guide is n \ge 30), or, for a small sample from a population of unknown shape, that sample’s plot shows no strong skew and no outliers. The two groups may meet it in different ways. Then
Why it is true
Center. The mean of a difference is the difference of the means. Since \mu_{\hat{p}_1} = p_1 and \mu_{\hat{p}_2} = p_2, the difference centers at p_1 - p_2; since \mu_{\bar{x}_1} = \mu_1 and \mu_{\bar{x}_2} = \mu_2, the difference of means centers at \mu_1 - \mu_2. No condition is needed beyond random sampling.
Spread. For independent random variables, the variance of a difference is the sum of the variances (Combining random variables). Subtracting \hat{p}_2 flips its errors in sign, but a variance squares each error, so a flipped error adds as much variance as an unflipped one. Each sample share has variance p(1-p)/n, so
and the same step with \sigma^2/n for each sample mean gives the Day 2 formula. Independence of the two samples is what licenses the addition, and the ten percent condition is what makes each sample’s own variance formula hold.
Shape. A difference of two independent normal variables is normal (Normal random variables). So when each statistic is normal, or nearly so, the difference is too.
Worked examples
Normal areas use z rounded to two decimal places and \Phi(z), the area to the left of z under the standard normal curve. For a negative z, \Phi(-z) = 1 - \Phi(z).
| z | \Phi(z) | z | \Phi(z) |
|---|---|---|---|
| 0.80 | 0.7881 | 1.90 | 0.9713 |
| 1.19 | 0.8830 | 2.25 | 0.9878 |
| 1.20 | 0.8849 | 2.37 | 0.9911 |
| 1.23 | 0.8907 | 2.39 | 0.9916 |
| 1.50 | 0.9332 | 2.81 | 0.9975 |
Cutoffs: \Phi(1.282) = 0.900 and \Phi(1.960) = 0.975.
Day 1: center, spread and conditions for the cash shares
Pema surveys a random sample of 120 Riverside shoppers and, separately, a random sample of 100 Hilltop shoppers, and records whether each pays cash. Describe the sampling distribution of \hat{p}_1 - \hat{p}_2.
SolutionCenter. The difference of sample shares centers at the difference of the true shares:
Conditions for the spread. The two samples are random and drawn separately, so they are independent of each other. Within each, the ten percent condition holds: 120 \le 0.10(5{,}000) = 500 and 100 \le 0.10(3{,}200) = 320.
Spread. Each share contributes its own variance:
so \displaystyle \sigma_{\hat{p}_1 - \hat{p}_2} = \sqrt{0.0044625} \approx 0.0668.
Shape. The expected counts are 120(0.55) = 66 and 120(0.45) = 54 at Riverside, 100(0.40) = 40 and 100(0.60) = 60 at Hilltop, all at least 10, so the difference is approximately normal.
In context: over all pairs of samples Pema could draw, the Riverside sample share exceeds the Hilltop share by 0.15 on average, and a typical difference misses 0.15 by about 0.067, nearly 7 percentage points. Each share alone has standard deviation \displaystyle \sqrt{0.0020625} \approx 0.0454 or \displaystyle \sqrt{0.0024} \approx 0.0490; the difference varies more than either, and less than their sum 0.0944.
Day 1: a small gap, and the wrong order
With Pema’s samples, find the chance that the Riverside sample share exceeds the Hilltop share by less than 0.05, and the chance that the Hilltop sample share comes out larger.
SolutionThe difference is approximately normal with mean 0.15 and standard deviation 0.0668, from Example 1.
A gap under 0.05. Standardize against the distribution of the difference:
so P(\hat{p}_1 - \hat{p}_2 < 0.05) \approx 1 - 0.9332 = 0.0668. The shaded tail of Figure 1 is this area.
The wrong order. The Hilltop share is larger exactly when the difference is negative:
so P(\hat{p}_1 - \hat{p}_2 < 0) \approx 1 - 0.9878 = 0.0122. A real gap of 15 percentage points leaves samples of this size ranking the markets the wrong way around only about once in 80 pairs of surveys, while a gap that looks much smaller than it is, under 5 points, turns up about 7 times in 100.
Day 1: choosing equal sample sizes
The association wants Pema to sample the same number n of shoppers at each market, enough that \sigma_{\hat{p}_1 - \hat{p}_2} is at most 0.04. Find the smallest such n, and check that the conditions still hold.
SolutionWith n_1 = n_2 = n the two variances share a denominator:
A standard deviation of at most 0.04 means a variance of at most 0.04^2 = 0.0016. Both sides are positive, so multiplying through keeps the direction:
A sample size is a whole number, so n = 305. Check: \displaystyle \sqrt{0.4875/305} \approx 0.03998, while n = 304 gives about 0.04005, just over. The ten percent condition at Hilltop needs 305 \le 320, which holds with little to spare; a sample of 305 at Riverside is well under 500. The large counts only grow with n. Precision costs sample size quickly: the spread falls only like one over \displaystyle \sqrt{n}, so cutting it from 0.0668 to 0.04 took about three times as many shoppers.
Day 2: center, spread and conditions for spending
Dolma records the amount spent by a random sample of 50 Riverside shoppers and, separately, a random sample of 40 Hilltop shoppers. Describe the sampling distribution of \bar{x}_1 - \bar{x}_2.
SolutionCenter. \mu_{\bar{x}_1 - \bar{x}_2} = 32 - 27 = 5 dollars.
Independence. Two separate random samples, and the ten percent condition holds for each: 50 \le 500 and 40 \le 320.
Spread. Each sample mean contributes \sigma^2/n:
so \displaystyle \sigma_{\bar{x}_1 - \bar{x}_2} = \sqrt{6.945} \approx 2.635 dollars, about 2.64.
Shape. Spending is not likely to be normal: no one spends less than 0, Riverside’s mean is only about 2.3 standard deviations above 0, and a few shoppers buy far more than most. But n_1 = 50 and n_2 = 40 are both past the usual guide of 30, so the The central limit theorem makes each sample mean, and therefore their difference, approximately normal.
In context: over all pairs of samples, the Riverside sample mean exceeds the Hilltop sample mean by 5 dollars on average, and a typical difference misses 5 dollars by about 2.64 dollars.
Day 2: when the Hilltop sample spends more
With Dolma’s samples, find the chance that the Hilltop sample mean is larger than the Riverside one, the chance that Riverside’s leads by more than 10 dollars, and the range that holds the middle 95\% of differences.
SolutionThe Hilltop mean is larger when \bar{x}_1 - \bar{x}_2 < 0. Using the normal model of Example 4:
so P(\bar{x}_1 - \bar{x}_2 < 0) \approx 1 - 0.9713 = 0.0287, the shaded tail of Figure 2.
A lead of more than 10 dollars is 5 dollars above the center, just as 0 is 5 dollars below it, so z \approx 1.90 and, by the symmetry of the normal curve, the chance is again 0.0287.
The middle 95\% runs 1.960 standard deviations each side of the center:
from about -0.17 to 10.17 dollars. Even with a true gap of 5 dollars, the middle 95\% of differences from samples of these sizes runs from about 0 to about 10 dollars, and about 3 pairs of samples in 100 put Hilltop ahead.
Day 2: small samples, and when that is allowed
Lhakpa buys loaves from the bakery stall at each market. Riverside loaves weigh \mu_1 = 24.4 ounces with \sigma_1 = 0.6; Hilltop loaves weigh \mu_2 = 24.0 ounces with \sigma_2 = 0.8; loaf weights at both stalls are normal. She weighs a random sample of 4 loaves from each stall. Find the chance that her Hilltop loaves average more than her Riverside loaves. Then say whether the same calculation could be done for the spending of only 12 shoppers at each market.
SolutionThe loaves. Both populations are normal, so each \bar{x} is exactly normal at any sample size, and so is their difference. Its center is 24.4 - 24.0 = 0.4 ounce, and
The Hilltop loaves average more when the difference is negative:
so the chance is 1 - 0.7881 = 0.2119. With only 4 loaves each, about one pair of samples in five ranks the stalls the wrong way around.
Spending, 12 shoppers each. The formulas still give the center, 5 dollars, and the spread, \displaystyle \sqrt{196/12 + 121/12} = \sqrt{26.4167} \approx 5.14 dollars. But the normal condition fails: spending is skewed right, as Example 4 argued, and 12 is far too few for the The central limit theorem to remove that skew. A normal model for \bar{x}_1 - \bar{x}_2 is not justified, so a z-score would give an untrustworthy chance. The normal condition is about each population’s shape and each sample’s size, never about the formulas for center and spread.
Boundaries
Two independent samples. The formulas add variances, which is only correct when the two statistics are independent. If Pema asked the same 100 shoppers about both markets, each answer pair would come from one person, the two shares would be linked, and \displaystyle \sqrt{\sigma_1^2/n_1 + \sigma_2^2/n_2} would give the wrong spread. Paired data are studied through the single list of differences instead.
The ten percent condition for each sample. A sample larger than a tenth of its own population makes that statistic vary less than its formula says. Pema’s 305 Hilltop shoppers in Example 3 are just inside the limit; a weekday market with only 400 shoppers could take no more than 40.
Normal shape for each group. Large counts (Day 1) or the normal condition (Day 2) must hold in both groups. One large sample does not rescue a small, skewed one: the difference inherits the skew of the worse-behaved statistic.
Known parameters. Every calculation here uses the true p_1, p_2, \sigma_1 and \sigma_2. When these are estimated from the samples, the same formulas become standard errors, and for means the standardized difference follows a The t distribution and degrees of freedom rather than the standard normal.
Where this goes next
This pattern is what turns a comparison of two samples into a statement about two populations. With the sample shares in place of p_1 and p_2, Day 1 becomes the Interval for a difference of proportions and the test for two proportions. With s_1 and s_2 in place of \sigma_1 and \sigma_2, Day 2 becomes the Interval for a difference of means and the two-sample t test. In an experiment, the two groups come from random assignment in a Completely randomized design and replication, and the same difference measures the effect of a treatment.
Exercises
Use the season records in the Definition, the \Phi table above, and independent random samples that meet the ten percent condition unless told otherwise. Population 1 is Riverside.
Day 1: a difference of two proportions
- The share of shoppers who bring their own bag is 0.70 at Riverside and 0.60 at Hilltop. For random samples of 80 Riverside and 60 Hilltop shoppers, find the mean and standard deviation of \hat{p}_1 - \hat{p}_2.
- Check the ten percent and large-counts conditions for Exercise 1.
- In the setting of Exercise 1, find P(\hat{p}_1 - \hat{p}_2 > 0.20).
- In the setting of Exercise 1, find the chance that the Riverside sample share of bag-bringers is larger than the Hilltop share.
- The share who buy flowers is 0.08 at Riverside and 0.15 at Hilltop. For samples of 100 at each market, find the mean and standard deviation of \hat{p}_1 - \hat{p}_2, and say whether a normal model may be used.
- For the cash shares, with samples of 300 Riverside and 250 Hilltop shoppers, find \sigma_{\hat{p}_1 - \hat{p}_2} and P(\hat{p}_1 - \hat{p}_2 < 0.05).
- In the setting of Exercise 1, find the range that holds the middle 95\% of values of \hat{p}_1 - \hat{p}_2.
- For the bag shares of Exercise 1, how many shoppers must be sampled at each market, the same number at both, for \sigma_{\hat{p}_1 - \hat{p}_2} to be at most 0.05?
- Hilltop’s weekday market has only 400 shoppers. Can a random sample of 60 of them be used with the formula for \sigma_{\hat{p}_1 - \hat{p}_2}? Explain.
Day 2: a difference of two means
- Time spent at the market has mean 48 minutes and standard deviation 20 minutes at Riverside, mean 40 minutes and standard deviation 15 minutes at Hilltop. For samples of 64 and 45 shoppers, find the mean and standard deviation of \bar{x}_1 - \bar{x}_2.
- In the setting of Exercise 10, find P(\bar{x}_1 - \bar{x}_2 > 12).
- In the setting of Exercise 10, find the chance that the Hilltop sample spends more time on average.
- For spending, with samples of 100 at each market, find \sigma_{\bar{x}_1 - \bar{x}_2} and P(\bar{x}_1 - \bar{x}_2 < 0).
- For spending with Dolma’s sample sizes, 50 and 40, find the value that \bar{x}_1 - \bar{x}_2 falls below only 10\% of the time.
- For Lhakpa’s loaves, with 9 loaves from each stall, find P(\bar{x}_1 - \bar{x}_2 < 0).
- Say whether a normal model for \bar{x}_1 - \bar{x}_2 is justified, and why: (a) spending, 35 Riverside and 45 Hilltop shoppers; (b) loaf weights, 3 loaves from each stall; (c) time spent, 20 Riverside and 50 Hilltop shoppers, where the Riverside sample’s dot plot shows one shopper far above the rest; (d) spending, 12 Riverside and 80 Hilltop shoppers.
- For spending, how many shoppers must be sampled at each market, the same number at both, for \sigma_{\bar{x}_1 - \bar{x}_2} to be at most 2 dollars?
Reasoning
- Pema says the standard deviation of \hat{p}_1 - \hat{p}_2 in Example 1 should be 0.0454 + 0.0490 = 0.0944. Explain why it is smaller.
- Explain why the means subtract but the variances add for a difference of independent statistics.
- Explain why the two samples must be independent for the formulas of this lesson, and give a survey of both markets that breaks this.
- Lhakpa takes 80 shoppers at one stall and randomly assigns half to taste a sample before buying. Explain why the ten percent condition is not checked for these two groups, and what condition replaces random sampling.
- Show that the formula for \sigma_{\bar{x}_1 - \bar{x}_2} does not depend on which population is called 1, and say what changes about the sampling distribution when the order is reversed.
- With equal sample sizes n and equal standard deviations \sigma, show that \sigma_{\bar{x}_1 - \bar{x}_2} is \displaystyle \sqrt{2} times the standard deviation of one sample mean.
Answers
- Mean 0.70 - 0.60 = 0.10; standard deviation \displaystyle \sqrt{\frac{0.21}{80} + \frac{0.24}{60}} = \sqrt{0.006625} \approx 0.0814.
- Ten percent: 80 \le 500 and 60 \le 320. Large counts: 80(0.70) = 56, 80(0.30) = 24, 60(0.60) = 36, 60(0.40) = 24, all at least 10. Both hold.
- 0.1093: \displaystyle z = \frac{0.20 - 0.10}{0.0814} \approx 1.23, and 1 - 0.8907 = 0.1093.
- 0.8907: the Riverside share is larger when the difference is positive; \displaystyle z = \frac{0 - 0.10}{0.0814} \approx -1.23, and 1 - 0.1093 = 0.8907.
- Mean 0.08 - 0.15 = -0.07; standard deviation \displaystyle \sqrt{\frac{0.0736}{100} + \frac{0.1275}{100}} \approx 0.0448. Not normal: 100(0.08) = 8 is under 10, so the large-counts condition fails at Riverside.
- \displaystyle \sqrt{\frac{0.2475}{300} + \frac{0.24}{250}} = \sqrt{0.001785} \approx 0.0422. Then \displaystyle z = \frac{0.05 - 0.15}{0.0422} \approx -2.37, so 1 - 0.9911 = 0.0089. (Ten percent: 300 \le 500, 250 \le 320.)
- From about -0.0595 to 0.2595: 0.10 \pm 1.96(0.0814) = 0.10 \pm 0.1595.
- 180 shoppers: \displaystyle \frac{0.21 + 0.24}{n} \le 0.05^2 = 0.0025 gives n \ge 180; and 180 \le 320 keeps the ten percent condition.
- No: 0.10(400) = 40, and 60 is more than that. Sampling so large a part of the population without replacement makes \hat{p}_2 vary less than its formula says, so the formula would overstate the spread.
- Mean 48 - 40 = 8 minutes; standard deviation \displaystyle \sqrt{\frac{400}{64} + \frac{225}{45}} = \sqrt{11.25} \approx 3.354 minutes.
- 0.1170: \displaystyle z = \frac{12 - 8}{3.354} \approx 1.19, and 1 - 0.8830 = 0.1170. Both samples exceed 30, so the normal model holds.
- 0.0084: \displaystyle z = \frac{0 - 8}{3.354} \approx -2.39, and 1 - 0.9916 = 0.0084.
- \displaystyle \sqrt{\frac{196}{100} + \frac{121}{100}} = \sqrt{3.17} \approx 1.780 dollars. \displaystyle z = \frac{0 - 5}{1.780} \approx -2.81, so 1 - 0.9975 = 0.0025.
- About 1.62 dollars: 5 - 1.282(2.635) = 1.6219.
- 0.1151: \displaystyle \sigma_{\bar{x}_1 - \bar{x}_2} = \sqrt{\frac{0.36}{9} + \frac{0.64}{9}} = \frac{1}{3}, \displaystyle z = \frac{0 - 0.4}{1/3} = -1.20, and 1 - 0.8849 = 0.1151.
- (a) Yes: both samples are past 30, so each mean is close to normal despite the skew. (b) Yes: both populations are normal, so any sample size works. (c) No: the Riverside sample is small and its plot has an outlier. (d) No: 12 Riverside shoppers are too few to overcome the skew of spending, and the large Hilltop sample does not fix that.
- 80 shoppers: \displaystyle \frac{196 + 121}{n} \le 2^2 = 4 gives n \ge 79.25; and 80 \le 320 keeps the ten percent condition.
- Standard deviations do not add; variances do. \displaystyle \sqrt{0.0454^2 + 0.0490^2} \approx 0.0668. Adding the standard deviations would assume both shares always miss in the widest direction at once, and independent misses seldom do.
- The center of a difference is found by subtracting the centers. Spread measures squared distances from the center, and subtracting a statistic flips the sign of its errors without changing their size, so its squared errors still add to the total; for independent statistics no cross term pulls the total down.
- The variance rule for a difference needs the two statistics to vary independently. A survey that asks the same shoppers about both markets links each Riverside answer to a Hilltop answer from the same person, so the two shares move together.
- No group is a sample from a larger population, so there is nothing for a tenth to be taken of. Random assignment of the shoppers to the two groups plays the part of random sampling: it makes the groups comparable and lets chance, not choice, decide who is in each.
- Swapping the samples swaps the two terms under the root, \displaystyle \frac{\sigma_2^2}{n_2} + \frac{\sigma_1^2}{n_1}, which is the same sum. The center changes sign, from \mu_1 - \mu_2 to \mu_2 - \mu_1, and the whole distribution is reflected about 0.
- \displaystyle \sqrt{\frac{\sigma^2}{n} + \frac{\sigma^2}{n}} = \sqrt{\frac{2\sigma^2}{n}} = \sqrt{2}\,\frac{\sigma}{\sqrt{n}}.