Interval for a difference of means
Two groups have different averages in their samples. How far apart are the means of the two whole populations, or, in an experiment, how much did one treatment change the mean response compared with the other?
Where you are 7 in Inference for means
When you reach for it
Zawadi grows two macadamia varieties on her orchard in Kenya, and a buyer pays more for heavier nuts. She wants to know how much heavier, on average, a variety A nut is than a variety B nut across this season’s whole harvest. She weighs a random sample of 20 variety A nuts, mean 7.8 grams, and an independent random sample of 18 variety B nuts, mean 7.0 grams (made-up data throughout this lesson). The gap in the samples is 0.8 grams, but another pair of samples would give another gap. What range of gaps between the two whole harvests do these samples support?
The two-sample interval for a difference of means answers that, for two populations sampled separately (Day 1) or two treatment groups in an experiment (Day 2). It is the One sample interval for a mean and sample size, \displaystyle \bar{x} \pm t^* s_x/\sqrt{n}, with two samples in place of one. Its standard error comes from Sampling distribution of a difference, whose Day 2 showed that for independent random samples \bar{x}_1 - \bar{x}_2 centers at \mu_1 - \mu_2 with standard deviation \displaystyle \sqrt{\sigma_1^2/n_1 + \sigma_2^2/n_2}. The parallel lesson for shares is the Interval for a difference of proportions.
The procedure
Day 1: two independent random samples
The population standard deviations are unknown, so the sample standard deviations stand in for them, and, as for one mean, that extra uncertainty calls for the The t distribution and degrees of freedom in place of the normal curve:
The square root is \text{SE}_{\bar{x}_1 - \bar{x}_2}, the Standard error of the difference, and t^* times it is the Point estimates and the margin of error.
Name the parameters. \mu_1 and \mu_2 are the means of a stated variable, with units, over two stated populations, and the interval is for \mu_1 - \mu_2. Saying which group is 1 fixes the order of subtraction.
Check the conditions for each sample. The Conditions for one mean apply to both samples, plus independence between them:
- Random and independent. Each sample is random from its own population, and the two samples are drawn independently. The variance of a difference is the sum of the two variances only for independent quantities. Two measurements on the same individuals, such as the same trees weighed in two seasons, are NOT two independent samples; they are Paired data: interval and test for a mean difference, a later lesson.
- Ten percent. Each sample is at most a tenth of its own population.
- Normal or large sample, for each group separately. A group passes if its population is close to normal, or its sample has n \ge 30 (unless the data are very strongly skewed), or a graph of its small sample shows no strong skew and no outliers. The two groups may pass in different ways, but both must pass.
When all three hold, the procedure is called a two-sample t interval for \mu_1 - \mu_2.
Degrees of freedom. The standardized difference does not follow a t curve exactly, but one particular t curve fits it very closely. Its degrees of freedom come from a formula due to Welch, which software and calculators use:
It is shown so the calculator’s number is not a mystery, and is rarely worked by hand; its value is usually not a whole number. By hand, the conservative choice is the smaller of n_1 - 1 and n_2 - 1. The Welch value is never below that, so the conservative interval is at least as wide, erring on the safe side.
Compute and conclude in context. The sentence names the level, both ends with units, both populations and the variable, and says which mean is larger: “We are 95\% confident that the mean ... of population 1 is between ... and ... more than that of population 2.” What the level promises is the subject of Confidence level and interpreting an interval.
Where 0 sits. A difference of 0 means equal means. An interval wholly above 0 is evidence, at that level, that \mu_1 > \mu_2; wholly below, that \mu_1 < \mu_2. An interval containing 0 makes equal means plausible, along with differences of either sign: no evidence of a difference, and no evidence that there is none.
Technology. A calculator’s 2-SampTInt takes the six summary numbers (or two data lists), the level, and a “Pooled” choice, which should be No. It prints the interval and the Welch degrees of freedom, subtracts in the order entered, and checks no condition.
Day 2: two treatments in a randomized experiment
In an experiment one set of subjects is split at random into two treatment groups, as in a Completely randomized design and replication. Here \mu_1 and \mu_2 are the mean responses if all the subjects received treatment 1, or all received treatment 2. The same formula and degrees of freedom apply. The random condition becomes Random assignment and confounding, whose chance is what makes \bar{x}_1 - \bar{x}_2 vary; the ten percent condition drops out, since nothing is sampled; the normal or large-sample condition is checked for each group as before.
The conclusion is about cause. Random assignment makes the groups alike, apart from chance, in everything but the treatment. So an interval wholly on one side of 0 is evidence that the treatment caused the difference, for the subjects in the experiment. Two sampled populations show a difference, never its cause.
The order of subtraction. Writing \mu_2 - \mu_1 instead of \mu_1 - \mu_2 negates the estimate and turns an interval from a to b into one from -b to -a. The standard error, the width and the conclusion stay the same; only the wording follows the sign.
Why the two variances are not pooled. Some books offer a pooled interval that assumes \sigma_1 = \sigma_2, merges s_1 and s_2 into one estimate, and uses n_1 + n_2 - 2 degrees of freedom. This course does not use it. Equal population spreads can rarely be checked, and when the spreads and sample sizes differ, the pooled interval’s capture rate can fall well short of its stated level. The unpooled interval stays close to its level either way, and when the spreads really are equal it is only a little wider. (The two-proportion test pools for a different reason: there the null hypothesis itself makes the two shares equal.)
Worked examples
Day 1: two macadamia varieties
Zawadi drew each sample at random from that variety’s harvest of tens of thousands of nuts. Variety A: n_A = 20, \bar{x}_A = 7.8 g, s_A = 1.1 g. Variety B: n_B = 18, \bar{x}_B = 7.0 g, s_B = 1.4 g. Dot plots of both samples are roughly symmetric with no outliers. Construct and interpret a 95\% confidence interval for the difference in mean nut weight, using the conservative degrees of freedom.
SolutionParameters. Let \mu_A be the mean weight of all variety A nuts in this season’s harvest and \mu_B the same for variety B. The interval is for \mu_A - \mu_B.
Conditions. Each sample is random from its own variety’s harvest, drawn separately, so random and independent. Both 20 and 18 are far below a tenth of tens of thousands. Both samples are under 30, so each needs its own graph, and both dot plots are clean. All conditions hold.
Compute. The conservative degrees of freedom are the smaller of 19 and 17, so \text{df} = 17 and, for 95\%, t^* = 2.110.
The interval is 0.8 \pm 0.868, from -0.068 to 1.668 grams, or about -0.07 to 1.67 g.
Conclude. We are 95\% confident that the mean weight of variety A nuts is between 0.07 g less and 1.67 g more than the mean weight of variety B nuts. The interval contains 0 (Figure 1, top bar). The samples fit A being heavier by over a gram, and also fit equal means or B slightly heavier. They do not show that A is heavier on average, and they do not show the varieties are equal either.
Day 1: reading 2-SampTInt output
Juma enters Zawadi’s six numbers into 2-SampTInt with level 0.95 and Pooled: No.
| 2-SampTInt | |
|---|---|
| interval | (-0.0381,\ 1.6381) |
| df | 32.24 |
| \bar{x}_1, \bar{x}_2 | 7.8, 7.0 |
| s_1, s_2 | 1.1, 1.4 |
| n_1, n_2 | 20, 18 |
Confirm the degrees of freedom from the Welch formula, and explain why this interval differs from the one in the previous example.
SolutionWith the two terms from the previous example, s_1^2/n_1 = 0.0605 and s_2^2/n_2 \approx 0.108889:
which matches. The center, \displaystyle \frac{-0.0381 + 1.6381}{2} = 0.8, and the standard error are as before; only t^* changed. With 32.24 degrees of freedom, t^* \approx 2.036, smaller than 2.110, so the margin is 2.036 \times 0.41157 \approx 0.838, half the width \displaystyle \frac{1.6381 - (-0.0381)}{2} = 0.8381. The technology interval is slightly narrower and more accurate, and still contains 0 (Figure 1, middle bar), so the conclusion stands.
Day 1: larger samples, and where 0 sits
Makena, a buyer at the local cooperative, thinks Zawadi’s samples were too small to settle the question. Next season she draws independent random samples the same way: 50 variety A nuts with \bar{x}_A = 7.7 g and s_A = 1.2 g, and 45 variety B nuts with \bar{x}_B = 7.1 g and s_B = 1.3 g. Construct a 95\% interval for \mu_A - \mu_B with conservative degrees of freedom, and compare it with technology’s interval, which uses \text{df} \approx 89.89 and prints (0.0882,\ 1.1118).
SolutionConditions. Random and independent as before; 50 and 45 are tiny next to the harvests; both samples have at least 30 nuts, and nut weights are not very strongly skewed, so the normal condition holds without a graph.
Compute. Conservative \text{df} = 44, the smaller of 49 and 44, and t^* = 2.015.
The interval is 0.6 \pm 0.519, from 0.081 to 1.119 g. Technology’s interval, 0.088 to 1.112, is almost the same: with samples this large the two choices of degrees of freedom give nearly equal t^*.
Conclude. We are 95\% confident that variety A nuts in this harvest weigh on average between 0.08 and 1.12 grams more than variety B nuts. The whole interval lies above 0, so these samples are evidence that A is heavier on average. The larger samples cut the standard error from about 0.41 to about 0.26, and the narrower interval no longer reaches 0 (Figure 1, bottom bar).
Day 2: a mulch trial
Juma tests a mulch of macadamia husks on his variety A trees. He takes 30 of his trees, assigns 15 at random to receive the mulch and the other 15 to his usual practice, and at harvest records each tree’s mean nut weight. Mulched trees: \bar{x}_M = 8.2 g, s_M = 0.6 g. Usual trees: \bar{x}_U = 7.6 g, s_U = 0.7 g. Dot plots of both groups show no strong skew and no outliers. Construct and interpret a 95\% interval for the difference in means, with conservative degrees of freedom.
SolutionParameters. Let \mu_M be the mean nut weight per tree if all 30 trees were mulched, and \mu_U the mean if all 30 had the usual practice. The interval is for \mu_M - \mu_U.
Conditions. The treatments were assigned at random, which is the random condition for an experiment. No population was sampled, so there is no ten percent check. Each group has 15 trees, under 30, and each group’s dot plot shows no strong skew and no outliers.
Compute. Conservative \text{df} = 14 and t^* = 2.145.
The interval is 0.6 \pm 0.511, from 0.089 to 1.111 grams. (Technology uses \text{df} \approx 27.36 and gives 0.112 to 1.088.)
Conclude. We are 95\% confident that, for these 30 trees, mulching raises the mean nut weight per tree by between 0.09 and 1.11 grams compared with the usual practice. The interval lies above 0 and the mulch was assigned at random, so this is evidence that the mulch causes heavier nuts on these trees. They are Juma’s own trees, not a random sample, so the result reaches other trees only as far as they resemble his.
Day 2: the other order, and a higher level
Zawadi rewrites Juma’s result as \mu_U - \mu_M. Find that interval, and find the 99\% interval for \mu_M - \mu_U with conservative degrees of freedom. Does either change the conclusion?
Reversing the order negates the estimate, \bar{x}_U - \bar{x}_M = -0.6, and leaves the standard error alone, since each term under the root is unchanged. The interval is -0.6 \pm 0.511, from -1.111 to -0.089 grams: the old interval flipped across 0, as Figure 2 shows. It lies wholly below 0, so the usual practice gives lighter nuts, by 0.09 to 1.11 grams. Same fact, told from the other side.
At 99\% with 14 degrees of freedom, t^* = 2.977:
so the interval for \mu_M - \mu_U is 0.6 \pm 0.709, from -0.109 to 1.309 grams. It now contains 0. At 95\% the trial shows the mulch helps; at 99\% it cannot rule out no effect. The level is chosen before the data are seen, never adjusted afterward to reach a wanted answer.
Day 2: same arithmetic, no cause
Makena compares nuts delivered to the cooperative from irrigated farms with nuts from rain-fed farms. From each season’s deliveries, tens of thousands of nuts of each kind, she weighs independent random samples: 40 nuts from irrigated farms, \bar{x}_I = 8.0 g, s_I = 1.0 g, and 35 from rain-fed farms, \bar{x}_R = 7.2 g, s_R = 1.2 g. Construct a 95\% interval for \mu_I - \mu_R with conservative degrees of freedom, and say whether it shows that irrigation causes heavier nuts.
SolutionConditions. Two independent random samples, each tiny next to its population, each with at least 30 nuts.
Compute. Conservative \text{df} = 34 and t^* = 2.032.
The interval is 0.8 \pm 0.523, from 0.277 to 1.323 grams.
Conclude. We are 95\% confident that nuts from irrigated farms weigh on average between 0.28 and 1.32 grams more than nuts from rain-fed farms. That is evidence of a real difference between the two populations, but not that irrigation causes it: farmers chose whether to irrigate, and irrigated farms may also differ in soil, altitude or care. The formula is the same as in Juma’s trial; what the interval can claim comes from how the groups were formed.
Boundaries
- The two samples must be independent. Measurements on the same individuals or matched pairs move together, and the sum-of-variances standard error is then wrong (Paired data: interval and test for a mean difference).
- Each group needs randomness, from random sampling or random assignment, and each must meet the normal or large-sample condition on its own; a large sample in one group does not rescue a small, strongly skewed one in the other.
- The ten percent condition applies to each sample separately, and not at all to an experiment with no sampling.
- The t curve is an approximation: the Welch degrees of freedom are the closer one, the conservative value a slightly wider one.
- A cause can be claimed only from random assignment, and only for the subjects unless they were randomly sampled.
- An interval that contains 0 is not evidence that the means are equal. The margin covers chance variation only, not a biased scale or a poor sampling frame.
Where this goes next
The same samples can be asked a yes-or-no question instead: is the difference real? That is the Test for a difference of means, which starts from \mu_1 - \mu_2 = 0 with the same standard error and degrees of freedom; a 95\% interval that excludes 0 and a two-sided test at the 0.05 level reach the same verdict. When both columns come from the same individuals, the work moves to Paired data: interval and test for a mean difference, which reduces the pairs to one column of differences.
Exercises
Use the conservative degrees of freedom unless told otherwise: for 95\%, t^* = 2.080 (21 df), 2.201 (11 df); for 90\%, t^* = 1.691 (34 df), 1.699 (29 df); for 99\%, t^* = 2.947 (15 df), 2.756 (29 df). All data are made up. Unless told otherwise, samples are independent random samples from harvests of tens of thousands of nuts, and the normal condition holds.
Day 1: two independent random samples
- Kernel weights: 25 variety A kernels have \bar{x} = 2.6 g and s = 0.40 g; 22 variety B kernels have \bar{x} = 2.3 g and s = 0.50 g. Name the parameters, and construct and interpret a 95\% interval for \mu_A - \mu_B.
- Nut diameters: 35 variety A nuts have \bar{x} = 22.4 mm and s = 1.8 mm; 40 variety B nuts have \bar{x} = 21.6 mm and s = 2.1 mm. Construct and interpret a 90\% interval for \mu_A - \mu_B.
- Shell thickness: 16 variety A nuts have \bar{x} = 3.1 mm and s = 0.5 mm; 16 variety B nuts have \bar{x} = 3.4 mm and s = 0.6 mm. Construct a 99\% interval for \mu_A - \mu_B and say what it shows.
- For each plan, say whether the two-sample t interval may be used, and if not, which condition fails. (a) Samples of 12 and 14 nuts; the dot plot of the 12 shows one nut at 13 g, far from all the others. (b) Nuts taken from the tops of the sacks. (c) A random sample of 40 nuts from a small trial harvest of 300 nuts, compared with a random sample of 40 from a large harvest. (d) Samples of 35 and 40 nuts whose histograms are mildly skewed, with no outliers.
- A 2-SampTInt output reads: interval (0.214,\ 1.386), \text{df} = 41.6, \bar{x}_1 = 8.3. Find the estimate, the margin of error and \bar{x}_2. Does the interval support a claim that group 1 averages at least 0.5 g more?
- For Exercise 1, compute the Welch degrees of freedom. Technology’s t^* for that value is 2.021; find the technology interval and say why it is narrower.
- For Makena’s larger samples (Example 3), the separate 95\% one-sample intervals use t^* = 2.010 (49 df) for variety A and t^* = 2.015 (44 df) for variety B. Compute both, show that they overlap, and say why that does not contradict the difference interval lying above 0.
- A report gives a 95\% interval for \mu_A - \mu_B from -0.15 to 0.95 g. Find the estimate and the margin, and say whether 0 and 1 are plausible values.
- Zawadi weighs one nut from each of 20 trees in one season, and one nut from each of the same 20 trees the next season. Can she use the two-sample t interval to compare the seasons? Explain.
Day 2: randomized experiments
- Makena assigns 24 mature trees at random, 12 to a new pruning method and 12 to no pruning. Yield per tree: pruned \bar{x} = 14.2 kg, s = 2.0 kg; unpruned \bar{x} = 12.1 kg, s = 2.4 kg. Dot plots show no strong skew or outliers. Construct and interpret a 95\% interval for \mu_{\text{pruned}} - \mu_{\text{none}}.
- In Exercise 10, give the interval for \mu_{\text{none}} - \mu_{\text{pruned}} and its conclusion.
- Juma assigns 60 sacks of nuts at random, 30 to sun drying and 30 to a solar dryer, and measures the moisture left, in percent. Sun: \bar{x} = 3.6, s = 0.5; dryer: \bar{x} = 3.2, s = 0.4. Construct 90\% and 99\% intervals for \mu_{\text{sun}} - \mu_{\text{dryer}} and say what each shows.
- For each study, say whether a 95\% interval lying above 0 would show cause, and why. (a) Two fertilizers are assigned at random to trees. (b) Random samples of nuts from young trees and from old trees are compared. (c) Trees whose owners chose to install drip lines are compared with a random sample of other trees.
- An experiment reports a 95\% interval for \mu_{\text{usual}} - \mu_{\text{new}} from -2.4 to -0.6 kg per tree. Which treatment gave the larger mean yield, and by how much?
- Makena’s pruning trial used 24 trees on one farm. Explain what would have to change for the conclusion to cover every mature tree in her district.
Reasoning
- Explain why the conservative degrees of freedom give an interval at least as wide as the technology interval, and why that is the safe direction to err.
- Show that the Welch degrees of freedom always lie between the smaller of n_1 - 1 and n_2 - 1 and n_1 + n_2 - 2.
- Zawadi’s first interval contained 0. Explain why she may not write that “the two varieties have the same mean nut weight.”
- Explain why the standard error formula fails for Exercise 9’s design.
- Explain in your own words why this course does not pool s_1 and s_2.
Answers
- \mu_A, \mu_B: mean kernel weights of all variety A and all variety B kernels. Estimate 0.3 g, \displaystyle \text{SE} = \sqrt{0.0064 + 0.011364} \approx 0.1333, \text{df} = 21, margin 2.080 \times 0.13328 \approx 0.2772, interval 0.0228 to 0.5772 g. We are 95\% confident variety A kernels weigh on average 0.02 to 0.58 g more; above 0, so evidence A is heavier.
- Estimate 0.8 mm, \displaystyle \text{SE} = \sqrt{0.092571 + 0.11025} \approx 0.4504, \text{df} = 34, margin 1.691 \times 0.45036 \approx 0.7616, interval 0.0384 to 1.5616 mm. We are 90\% confident variety A nuts are on average 0.04 to 1.56 mm wider.
- Estimate -0.3 mm, \displaystyle \text{SE} = \sqrt{0.015625 + 0.0225} \approx 0.1953, \text{df} = 15, margin 2.947 \times 0.19526 \approx 0.5754, interval -0.8754 to 0.2754 mm. It contains 0: at 99\% the samples give no evidence that the mean shell thicknesses differ.
- (a) No: the sample of 12 is small and has an outlier, so its normal condition fails. (b) No: not a random sample; nuts on top may differ from the rest. (c) No: 40 is more than a tenth of 300, which is 30. (d) Yes: both samples have at least 30, and mild skew is fine.
- Estimate \displaystyle \frac{0.214 + 1.386}{2} = 0.8 g, margin \displaystyle \frac{1.386 - 0.214}{2} = 0.586 g, \bar{x}_2 = 8.3 - 0.8 = 7.5 g. No: values from 0.214 to 0.5 are inside the interval, so a gap smaller than 0.5 g is plausible.
- \displaystyle \text{df} = \frac{(0.017764)^2}{0.0064^2/24 + 0.011364^2/21} \approx 40.17. Interval 0.3 \pm 2.021 \times 0.13328, which is 0.3 \pm 0.2694, from 0.0306 to 0.5694 g. More degrees of freedom give a smaller t^* (2.021 against 2.080), so the margin is smaller.
- Variety A: \displaystyle 7.7 \pm 2.010 \times 1.2/\sqrt{50}, margin \approx 0.3411, from 7.3589 to 8.0411 g. Variety B: \displaystyle 7.1 \pm 2.015 \times 1.3/\sqrt{45}, margin \approx 0.3905, from 6.7095 to 7.4905 g. They overlap from 7.3589 to 7.4905. The difference’s standard error is \displaystyle \sqrt{a^2 + b^2}, smaller than a + b, so overlap is the wrong test of a difference.
- Estimate \displaystyle \frac{-0.15 + 0.95}{2} = 0.4 g, margin \displaystyle \frac{0.95 - (-0.15)}{2} = 0.55 g. 0 is inside, so plausible; 1 is above 0.95, so not plausible.
- No. The same trees are measured twice, so a heavy-nut tree tends to be heavy in both seasons and the two samples are not independent. This is Paired data: interval and test for a mean difference: take the 20 differences and use a one-sample interval on them.
- Random assignment; 12 per group with clean dot plots. Estimate 2.1 kg, \displaystyle \text{SE} = \sqrt{0.33333 + 0.48} \approx 0.9018, \text{df} = 11, margin 2.201 \times 0.90185 \approx 1.985, interval 0.115 to 4.085 kg. We are 95\% confident that for these 24 trees the pruning raises mean yield by 0.115 to 4.085 kg per tree; random assignment makes this evidence of cause.
- From -4.085 to -0.115 kg: unpruned trees yield less, by 0.115 to 4.085 kg per tree. Same conclusion, same width.
- Estimate 0.4 points, \displaystyle \text{SE} = \sqrt{0.008333 + 0.005333} \approx 0.1169, \text{df} = 29. At 90\%: margin 1.699 \times 0.11690 \approx 0.1986, interval 0.2014 to 0.5986. At 99\%: margin 2.756 \times 0.11690 \approx 0.3222, interval 0.0778 to 0.7222. Both lie above 0: with random assignment, this is evidence that the dryer causes drier nuts for these sacks.
- (a) Yes: random assignment balances everything but the fertilizer. (b) No: tree age cannot be assigned, and old and young trees may differ in many other ways. (c) No: owners chose drip lines, so the care and wealth behind that choice are mixed up with the drip lines.
- Every plausible value of \mu_{\text{usual}} - \mu_{\text{new}} is negative, so the new treatment gave the larger mean yield, by between 0.6 and 2.4 kg per tree.
- The 24 trees would have to be a random sample of the district’s mature trees (with the ten percent condition checked), and the pruning then assigned at random within that sample.
- The Welch value is never below the smaller of n_1 - 1 and n_2 - 1, and fewer degrees of freedom mean heavier tails and a larger t^*. A wider interval captures the true difference at least as often as stated, so it never claims more confidence than it has.
- Write a = s_1^2/n_1, b = s_2^2/n_2, k_1 = n_1 - 1, k_2 = n_2 - 1, k = \min(k_1, k_2). Since a^2/k_1 + b^2/k_2 \le (a^2 + b^2)/k \le (a + b)^2/k, the df is at least k. By the Cauchy–Schwarz inequality, (a + b)^2 \le (k_1 + k_2)(a^2/k_1 + b^2/k_2), so the df is at most k_1 + k_2 = n_1 + n_2 - 2.
- The interval contains 0, but also every value up to 1.67 g. A one-gram gap fits the data as well as no gap, so a difference is not shown, and neither is equality.
- Adding the two variances is right only for independent sample means. Measurements on the same trees are linked, so the true spread of the difference is not \displaystyle \sqrt{s_1^2/n_1 + s_2^2/n_2}.
- Pooling assumes equal population standard deviations, which can rarely be checked; if they differ, especially with unequal sample sizes, the pooled interval can capture the truth less often than stated. The unpooled interval holds its level either way and costs little width when the spreads are equal.