Statistics

The ten percent condition

Picking people one at a time without putting anyone back changes the odds on every later pick. When is that change small enough to ignore?

You need

Sampling variability and simulating itMath 7 Compound eventsGrades 7 to 11

Unlocks

Conditions for one meanStatistics Conditions for one proportionStatistics Goodness of fit testStatistics

Where you are 7 in Sampling distributions

Statistics7

Distributions8

Sampling distributions7

▴ 1 above Sampling variability and simulating it Simulating a sampling distribution and a margin of error

The ten percent condition

Sampling distribution of a sample mean Sampling distribution of a sample proportion ▾ Show all 7 (1 more)

Why this exists

A mountain trail race has 1500 entrants, and 450 of them are from out of state (made-up data). A race official, Bishnu, wants to learn about the entrants by asking a random sample of them. He samples without replacement: once a runner is picked, that runner is set aside, because asking the same person twice tells him nothing new.

The formulas a statistics course builds next, the binomial distribution and the standard deviation of a sample proportion, assume the draws are independent: the chance of a success on each draw does not depend on what the earlier draws gave. Without replacement it does depend, since each pick removes a runner and changes the mix left for the next pick.

There are two ways out. One is to redo every formula for draws without replacement, which can be done (Probability with counting formulas does the exact counting) but is heavier. The other is a test that says when the independent formulas are close enough to the truth to use. The ten percent condition is that test.

Definition

In plain words: a random sample drawn without replacement may be treated as if its draws were independent when the sample is at most one tenth of the population.

With N the size of the population and n the size of the sample, the ten percent condition is

n \le 0.10\,N, \quad \text{or equivalently} \quad N \ge 10n.

The condition is checked only when sampling without replacement from a finite population. Draws with replacement, and the outcomes of a random process such as spinning a spinner, are independent already, and the condition has nothing to say about them.

The number 10\% is a convention, not a theorem. Draws without replacement are never exactly independent, at 10\% or at 1\%. What a theorem does give (the Results below) is the exact size of the error made by pretending they are, and at 10\% that error is small enough that statisticians have agreed to ignore it. A stricter field could draw the line lower.

Intuition

On Bishnu’s first pick the chance of an out-of-state runner is \displaystyle \frac{450}{1500} = 0.3. If that runner was from out of state, 449 out-of-state runners remain among 1499, so the chance on the second pick is \displaystyle \frac{449}{1499} \approx 0.29953; if not, it is \displaystyle \frac{450}{1499} \approx 0.30020. The second pick depends on the first, so the draws are dependent, but the dependence moves the chance only in the fourth decimal place. Removing one runner from 1500 barely changes the mix.

Each further pick removes another runner, and the small shifts pile up. After 400 picks the 1100 runners left can differ noticeably from the original mix. What decides how much the shifts matter is the fraction of the population already taken, \displaystyle \frac{n}{N}.

The cleanest measure of the total effect is a single number: the factor by which the true standard deviation of a sample proportion is smaller than the independent formula says. Figure 1 plots it for samples from the 1500 entrants.

Figure 1The finite population correction factor for samples drawn without replacement from the 1500 race entrants (made-up data). The dots mark samples of 50 and 400; the dashed vertical line marks 10\% of the population.

For small samples the curve hugs 1: they behave like independent draws. It sinks slowly, reaches about 0.95 at the 10\% line, then falls faster and reaches 0 at n = N. There the sample is the whole population, so the sample proportion equals p every time and has no spread at all.

Results

Sampling without replacement. Take a simple random sample of size n, without replacement, from a population of size N in which the share of successes is p. The sample proportion \hat{p} has mean p, exactly as for independent draws, and standard deviation

\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}} \cdot \sqrt{\frac{N-n}{N-1}}.

The first factor is the standard deviation for independent draws. The second is the finite population correction factor. The same factor multiplies the standard deviation \displaystyle \frac{\sigma}{\sqrt{n}} of a sample mean.

What the condition buys. For n \ge 1 the correction factor is at most 1 (and below 1 once n \ge 2), so the independent formula never understates the spread; for any sample of two or more it overstates it. When n \le 0.10\,N,

\sqrt{\frac{N-n}{N-1}} \ge \sqrt{0.9} \approx 0.949,

so under the ten percent condition the independent formula is at most about 5\% too large.

Why it is true

Write the count of successes as X = X_1 + X_2 + \cdots + X_n, where X_i = 1 if draw i is a success and 0 if not. Before anything is known about the other draws, draw i is equally likely to be any member of the population, so P(X_i = 1) = p. Then E[X_i] = p and \operatorname{Var}(X_i) = E[X_i^2] - p^2 = p - p^2 = p(1-p). The mean of X is np, so the mean of \displaystyle \hat{p} = \frac{X}{n} is p, with or without replacement.

For the spread we need the general rule for the variance of a sum, which holds whether or not the parts are independent. It is used here without proof (it comes from expanding the square inside the definition of variance), and it adds a term for every ordered pair of different draws:

\begin{aligned} \operatorname{Var}(X) &= \sum_{i} \operatorname{Var}(X_i) \\ &\quad + \sum_{i \ne j} \Big( E[X_i X_j] - E[X_i]\,E[X_j] \Big). \end{aligned}

For independent draws every pair term is 0, which is the rule variances add. Without replacement, E[X_i X_j] is the chance that both draws are successes. The population holds Np successes, so by the The general multiplication rule that chance is \displaystyle \frac{Np}{N} \cdot \frac{Np - 1}{N - 1}, and each pair term is

\begin{aligned} \frac{p(Np-1)}{N-1} - p^2 &= \frac{p^2 N - p - p^2 N + p^2}{N-1} \\ &= -\frac{p(1-p)}{N-1}. \end{aligned}

It is negative: a success taken out makes the next success a little less likely. With n(n-1) ordered pairs,

\begin{aligned} \operatorname{Var}(X) &= np(1-p) - \frac{n(n-1)\,p(1-p)}{N-1} \\ &= np(1-p) \cdot \frac{N-n}{N-1}. \end{aligned}

Dividing by n^2 gives \displaystyle \operatorname{Var}(\hat{p}) = \frac{p(1-p)}{n} \cdot \frac{N-n}{N-1}, and the square root is the result. For the race the pair term is \displaystyle -\frac{0.21}{1499} \approx -0.00014, the small gap Example 1 finds.

Finally, N - 1 < N, so \displaystyle \frac{N-n}{N-1} \ge \frac{N-n}{N} = 1 - \frac{n}{N}, and when \displaystyle \frac{n}{N} \le 0.10 this is at least 0.9.

Worked examples

Example 1

Two entrants, both from out of state

Bishnu picks two of the 1500 entrants at random, without replacement. Find the probability that both are from out of state, and compare it with the answer the draws would give if they were independent.

Solution

The second pick depends on the first, so the The general multiplication rule applies: multiply the chance for the first pick by the chance for the second given the first.

P(\text{both}) = \frac{450}{1500} \cdot \frac{449}{1499} \approx 0.08986.

If the draws were independent, the Independent events and the multiplication rule would give 0.3^2 = 0.09. The gap is about 0.00014, roughly 0.16\% of 0.09. Two picks are a tiny fraction of the population, \displaystyle \frac{2}{1500} \approx 0.13\%, and treating them as independent costs almost nothing.

Example 2

A small group, where it matters

At the start line the front row holds 10 entrants, 3 of them from out of state: the same share, 0.3. Sunita picks two of the ten at random without replacement. Find the probability that both are from out of state, and compare.

Solution

By the general multiplication rule,

P(\text{both}) = \frac{3}{10} \cdot \frac{2}{9} = \frac{6}{90} = \frac{1}{15} \approx 0.0667.

The independent answer, 0.09, is about 35\% too large. The share and the sample size are the same as in Example 1; what changed is the population. Two of ten is 20\% of the group, well past the ten percent condition, and the first pick changes the second pick’s chance from 0.3 to \displaystyle \frac{2}{9} \approx 0.222.

Example 3

A sample of 50

Bishnu surveys a random sample of 50 entrants, without replacement, and records \hat{p}, the share from out of state. Check the ten percent condition, then find the standard deviation of \hat{p} from the independent formula and from the exact one.

Solution

The sample is \displaystyle \frac{50}{1500} \approx 0.033 of the population, under 0.10, so the condition is met.

The independent formula gives \displaystyle \sqrt{\frac{0.3 \cdot 0.7}{50}} = \sqrt{0.0042} \approx 0.0648. The correction factor is \displaystyle \sqrt{\frac{1450}{1499}} \approx 0.9835, so the exact standard deviation is 0.0648 \cdot 0.9835 \approx 0.0637.

In context: samples of 50 entrants give out-of-state shares that typically miss 0.3 by about 0.064, roughly 6 percentage points. The independent formula is off by about 0.001, under 2\% of its value, which is the kind of error the condition promises to keep small.

Example 4

A sample of 400

Anil, another official, samples 400 entrants without replacement. Repeat Example 3 for his sample.

Solution

Now \displaystyle \frac{400}{1500} \approx 0.267, above 0.10: the condition fails.

The independent formula gives \displaystyle \sqrt{\frac{0.21}{400}} \approx 0.0229. The correction factor is \displaystyle \sqrt{\frac{1100}{1499}} \approx 0.8566, so the exact standard deviation is about 0.0196. The independent value is \displaystyle \frac{0.0229}{0.0196} \approx 1.17 times the truth, about 17\% too large.

So a margin of error built on the independent formula would be too wide: it would claim less precision than Anil’s sample really has. Compare the sample of exactly 10\%, n = 150, whose factor is \displaystyle \sqrt{\frac{1350}{1499}} \approx 0.949. The ten percent condition draws its line where the error is about 5\%.

Example 5

Checking the condition in four situations

Decide whether the ten percent condition is met, or needed at all.

(a) Sunita samples 120 entrants without replacement. (b) Sunita samples 200 entrants without replacement. (c) Anil’s computer picks 40 entrant numbers with replacement, so a runner may come up twice. (d) Bishnu samples 90 runners without replacement from everyone who entered any mountain trail race in the country last year, a number nobody has counted.

Solution

(a) \displaystyle \frac{120}{1500} = 0.08 \le 0.10: met.

(b) \displaystyle \frac{200}{1500} \approx 0.133 > 0.10: not met. The independent formulas would overstate the spread; use the correction factor instead.

(d) The condition asks for N \ge 10 \cdot 90 = 900. The population is not counted, but entrants to every such race in the country surely number far more than 900, so the condition is met. It is stated as an assumption about N, since N is not known.

Example 6

No out-of-state runners among five

Bishnu picks 5 entrants without replacement. Find the probability that none is from out of state, exactly, and then by treating the five draws as independent, which is what a binomial model with n = 5 and p = 0.3 does.

Solution

There are 1500 - 450 = 1050 in-state entrants. Exactly, each pick must be in state given that the earlier ones were:

\begin{aligned} &\frac{1050}{1500} \cdot \frac{1049}{1499} \cdot \frac{1048}{1498} \\ &\quad \cdot \frac{1047}{1497} \cdot \frac{1046}{1496} \approx 0.16759. \end{aligned}

Treating the draws as independent, each has chance 0.7 of being in state, so the answer is 0.7^5 = 0.16807. The two differ by about 0.0005. Five is a tiny share of 1500, the condition holds, and the binomial answer is close. This is the role the condition plays in the independence check of the binomial setting.

Boundaries

The condition applies only to sampling without replacement from a finite population. Draws with replacement and the outcomes of a random process are independent, and the condition is not checked for them.
It speaks to independence and nothing else. It does not make a sample random: a sample of 50 runners chosen because they finished first is biased however small it is. It also says nothing about the shape of the sampling distribution, which is a separate check (the large counts condition in Conditions for one proportion).
Nothing changes at exactly 10\%. A sample of 140 and a sample of 160 from the 1500 entrants have correction factors that differ by less than 0.01; the line is a convention placed where the error is about 5\%.
Failing the condition does not forbid the analysis. The correction factor gives the exact standard deviation for a simple random sample, and exact counting (see Probability with counting formulas) gives exact probabilities. The mean of \hat{p} is p either way.

Where this goes next

The Sampling distribution of a sample proportion and the Sampling distribution of a sample mean are first taught with the independent formulas \displaystyle \sqrt{\frac{p(1-p)}{n}} and \displaystyle \frac{\sigma}{\sqrt{n}}, and the ten percent condition is what licenses using them for real samples, which are almost always drawn without replacement. The same check is the independence condition of the binomial setting, so that the Mean and standard deviation of a binomial and the binomial probabilities apply to people drawn from a list. Every inference procedure that follows lists it among its conditions: Conditions for one proportion, Conditions for one mean and the Goodness of fit test. The Sampling variability and simulating it seen in simulations is, in this light, exactly the spread these formulas predict.

Exercises

Draws without replacement

Of the 1500 entrants, 600 are running their first trail race (made-up data). Bishnu picks two entrants at random without replacement. Find the exact probability that both are first-timers, and the value the independent rule would give.
At an aid station a group of 8 entrants rests, 2 of them aged 50 or over. Sunita picks two of the eight without replacement. Find the probability that both are aged 50 or over, and compare with the independent answer for a share of 0.25.
Of the 1500 entrants, 225 are aged 50 or over. Anil picks three entrants without replacement. Find the exact probability that none is aged 50 or over, and compare with 0.85^3.
Anil draws an entrant number from a box of all 1500, puts it back, and draws again. Find the probability that both draws are first-timers (Exercise 1’s share).

Checking the condition

Sunita samples 100 entrants without replacement. Is the ten percent condition met?
Bishnu samples 160 entrants without replacement. Is the condition met?
A valley trail race has 2340 entrants. What is the largest sample of them that meets the condition?
Sunita samples 75 people without replacement from all registered trail runners in a state, a number she does not know. What must be true for the condition to be met?
Bishnu draws 60 entrant numbers with replacement. Must he check the condition?

The correction factor

Find the correction factor for a sample of 100 from the 1500 entrants.
For a sample of 100 entrants, find the standard deviation of the sample share of first-timers (p = 0.4), first from the independent formula, then exactly.
Repeat Exercise 11 for a sample of 750, half the entrants. How many times the truth is the independent value?
Bishnu surveys all 1500 entrants. What is the correction factor, and what does it say about \hat{p}?
A big-city road race has 150{,}000 entrants. Find the correction factor for a sample of 1500 of them, and compare it with Exercise 13.
Find the largest sample size n from the 1500 entrants whose correction factor is at least 0.99.
Show that for the first-timers the pair term \displaystyle \frac{600}{1500} \cdot \frac{599}{1499} - 0.16 equals \displaystyle -\frac{0.24}{1499}, and give its value.

Reasoning

Using the chances on Bishnu’s second pick, explain why his draws without replacement are not independent, and why they are nearly so.
Show that \displaystyle \frac{N-n}{N-1} \le 1 whenever n \ge 1, and say what this means for the independent formula.
The 10\% line is a convention. If a field wanted the independent formula to be at most 1\% too large, roughly what share of the population could it sample?
Anil says a sample of 500 from the 150{,}000 entrants of Exercise 14 is “only a third of a percent of the field, so it can’t say much.” Explain what is wrong with his reasoning.

Answers

\displaystyle \frac{600}{1500} \cdot \frac{599}{1499} \approx 0.15984, against 0.4^2 = 0.16 if independent. The gap is about 0.00016.
\displaystyle \frac{2}{8} \cdot \frac{1}{7} = \frac{1}{28} \approx 0.0357, against 0.25^2 = 0.0625. The independent answer is far off because two of eight is 25\% of the group.
There are 1275 entrants under 50. Exactly, \displaystyle \frac{1275}{1500} \cdot \frac{1274}{1499} \cdot \frac{1273}{1498} \approx 0.61391; independently, 0.85^3 = 0.614125. They differ by about 0.0002.
0.4^2 = 0.16 exactly. With replacement each draw is made from all 1500, so the draws are independent and the multiplication rule for independent events applies.
Yes: \displaystyle \frac{100}{1500} \approx 0.067 \le 0.10.
No: \displaystyle \frac{160}{1500} \approx 0.107 > 0.10.
234, since 0.10 \cdot 2340 = 234.
The population must hold at least 10 \cdot 75 = 750 runners. She states this as an assumption, reasonable if the state has many registered trail runners.
No. With replacement the draws are independent, so the condition does not apply.
\displaystyle \sqrt{\frac{1400}{1499}} \approx 0.9664.
Independent: \displaystyle \sqrt{\frac{0.24}{100}} \approx 0.0490. Exact: 0.0490 \cdot 0.9664 \approx 0.0473.
Independent: \displaystyle \sqrt{\frac{0.24}{750}} \approx 0.0179. Factor \displaystyle \sqrt{\frac{750}{1499}} \approx 0.7073, so exact \approx 0.0127. The independent value is \displaystyle \frac{1}{0.7073} \approx 1.41 times the truth.
\displaystyle \sqrt{\frac{0}{1499}} = 0. A census gives \hat{p} = p every time, so \hat{p} has no spread.
\displaystyle \sqrt{\frac{148500}{149999}} \approx 0.9950. The same n = 1500 is only 1\% of this population, so it behaves like independent draws, while in Exercise 13 it is the whole population.
n = 30. The factor is at least 0.99 when \displaystyle \frac{1500 - n}{1499} \ge 0.9801, that is 1500 - n \ge 1469.17, so n \le 30.83. Check: n = 30 gives about 0.99028, and n = 31 gives about 0.98994.
Since \displaystyle \frac{600}{1500} = 0.4, the first term is \displaystyle \frac{0.4 \cdot 599}{1499} = \frac{239.6}{1499}, and \displaystyle 0.16 = \frac{0.16 \cdot 1499}{1499} = \frac{239.84}{1499}. So the pair term is \displaystyle \frac{239.6 - 239.84}{1499} = -\frac{0.24}{1499} \approx -0.00016, which is \displaystyle -\frac{p(1-p)}{N-1} with p = 0.4.
The second pick’s chance of an out-of-state runner is \displaystyle \frac{449}{1499} if the first was out of state and \displaystyle \frac{450}{1499} if not. These differ, so the second draw depends on the first. Both are within 0.0005 of 0.3, so the dependence is tiny: one runner is a very small part of 1500.
For n \ge 1, N - n \le N - 1, so the fraction is at most 1 and so is its square root. The exact standard deviation is never larger than the independent one: the independent formula can only overstate the spread, never understate it.
About 2\%. The independent value is at most 1.01 times the truth when the factor is at least \displaystyle \frac{1}{1.01} \approx 0.990, so \displaystyle \frac{N-n}{N-1} must be at least about 0.980, which for a large population means \displaystyle \frac{n}{N} at most about 0.02.
The precision of \hat{p} is set mainly by n, through \displaystyle \sqrt{\frac{p(1-p)}{n}}, not by the share of the population sampled. A small share only makes the correction factor nearly 1. A sample of 500 has the same standard deviation, within a fraction of a percent, whether the field is 150{,}000 or ten times larger.