Statistics · S.IC.B.4

Point estimates and the margin of error

A sample gives one number, and that number is almost surely a little off. How far off might it reasonably be, and what makes that distance bigger or smaller?

You need

Standard errorStatistics

Unlocks

Confidence level and interpreting an intervalStatistics

Where you are 4 in Confidence intervals

Statistics7

Statistical inference8

Confidence intervals4

Confidence level and interpreting an interval

Point estimates and the margin of error

Standard error Critical value

Why this exists

A city transit agency wants to know what share of all its riders are satisfied with the service. Mehwish, who runs the survey, cannot ask every rider, so she asks a random sample of 600 riders and 384 say they are satisfied (made-up data). The sample proportion is

\hat{p} = \frac{384}{600} = 0.64

Reporting “64\% of riders are satisfied” claims too much. Another random sample of 600 would give a slightly different share, and the true share p among all riders is almost surely not exactly 0.64. Reporting nothing but “somewhere between 0 and 1” claims too little. What the agency needs is the single best number together with an honest distance: how far that number might reasonably sit from the truth, given how much samples of this size vary.

The idea first appeared in Simulating a sampling distribution and a margin of error, where a margin came from the spread of a pile of simulated samples. Here there is no pile and no simulation, only the one sample in hand, and the spread comes from its Standard error.

Definition

A point estimate is a single value of a statistic, computed from the sample, used as the estimate of a parameter. The sample proportion \hat{p} is the point estimate of the population proportion p, and the sample mean \bar{x} is the point estimate of the population mean \mu. Both are unbiased estimators when the sample is random.

The margin of error is the distance, on each side of the point estimate, within which the estimate is likely to lie from the parameter, taking into account only the chance variation of random sampling. It is a multiple of the standard error:

\text{ME} = (\text{multiplier}) \times \text{SE}

and the estimate is reported as

\text{point estimate} \pm \text{ME}

which names a range of values from the point estimate minus the margin to the point estimate plus the margin. The margin of error is half the width of that range.

In this lesson the multiplier is 2: a margin of about 2 standard errors goes with about 95\% of samples. The exact multiplier for a chosen percentage is the Critical value, and what the percentage says about one particular range is the Confidence level and interpreting an interval. Both are later lessons.

Intuition

Recall from Standard error that the standard error is the sample’s estimate of the spread of the statistic’s sampling distribution: for a proportion \displaystyle \text{SE}_{\hat{p}} = \sqrt{\hat{p}(1 - \hat{p})/n} and for a mean \displaystyle \text{SE}_{\bar{x}} = s_x/\sqrt{n}. It is the typical distance between a statistic and the parameter. The margin of error stretches that typical distance to a distance that almost all samples stay inside.

Why 2? For a large random sample the sampling distribution of \hat{p} or \bar{x} is close to normal (the The central limit theorem), centered on the parameter. In a normal distribution about 95\% of values lie within 2 standard deviations of the center. So about 95\% of samples give an estimate within 2 standard errors of the parameter.

Figure 1The transit survey’s estimate (made-up data). The point estimate \hat{p} = 0.64 sits in the middle, and the margin of error, about 2 standard errors, reaches the same distance 0.039 to each side.

Figure 1 shows the transit survey’s result. The point estimate is the dot; the margin is the reach of the bar on each side. The bar does not say that the truth is spread out; p is one fixed number. It says how far a method like this one can miss.

Three things move the margin, and each is visible in the formula \displaystyle 2\sqrt{\hat{p}(1 - \hat{p})/n}:

Sample size. n sits under a square root in the denominator, so a larger sample gives a smaller margin, but slowly: 4 times the sample halves it (Figure 2).
The percentage wanted. Covering more samples than about 95\% takes a multiplier larger than 2, so a wider margin.
The proportion itself. For a fixed n, the factor \hat{p}(1 - \hat{p}) is largest at \hat{p} = 0.5, so a near-even split gives the widest margin.

Figure 2The margin of error \displaystyle 2\sqrt{0.64 \times 0.36 / n} as the sample size grows (made-up data). Each step from 150 to 600 to 2{,}400 riders multiplies the sample by 4 and halves the margin.

Results

For a random sample of size n, with a multiplier of 2 for about 95\%:

\text{ME}_{\hat{p}} \approx 2\sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}

\text{ME}_{\bar{x}} \approx 2 \cdot \frac{s_x}{\sqrt{n}}

The estimate is reported as \hat{p} \pm \text{ME} or \bar{x} \pm \text{ME}.

Multiplying the sample size by k divides the margin of error by \displaystyle \sqrt{k}. Four times the sample halves the margin; nine times cuts it to a third.

For any sample proportion, \hat{p}(1 - \hat{p}) \le 0.25, so

\text{ME}_{\hat{p}} \le 2\sqrt{\frac{0.25}{n}} = \frac{1}{\sqrt{n}}

with equality at \hat{p} = 0.5. A margin m is guaranteed, whatever the proportion turns out to be, by a sample of at least \displaystyle n = \frac{1}{m^2}, rounded up to a whole number.

A larger multiplier, for a higher percentage of samples, gives a proportionally larger margin.

Why it is true

Why the distance can be turned around. In about 95\% of samples, \hat{p} lands within 2 standard deviations of p. “The estimate is within 2 standard deviations of the parameter” and “the parameter is within 2 standard deviations of the estimate” are the same statement, because distance on a number line is the same in both directions. So for about 95\% of samples the range \hat{p} \pm 2\sigma_{\hat{p}} reaches p. The standard deviation \sigma_{\hat{p}} needs p, so the standard error, which is close to it for a large sample, takes its place.

Why four times the sample halves the margin. Replacing n by kn gives

\begin{aligned} 2\sqrt{\frac{\hat{p}(1 - \hat{p})}{kn}} &= \frac{1}{\sqrt{k}} \\ &\quad \cdot 2\sqrt{\frac{\hat{p}(1 - \hat{p})}{n}} \end{aligned}

and \displaystyle \frac{1}{\sqrt{4}} = \frac{1}{2}. The same holds for \displaystyle s_x/\sqrt{n}.

Why \hat{p} = 0.5 gives the largest margin. Completing the square,

\hat{p}(1 - \hat{p}) = \hat{p} - \hat{p}^2 = 0.25 - (\hat{p} - 0.5)^2

A square is never negative, so the product is at most 0.25, reached only when \hat{p} = 0.5. The square root is increasing, so the margin is largest there too.

Worked examples

Example 1

The rider survey

Mehwish’s random sample of 600 riders has 384 satisfied. Give the point estimate, its standard error, and the margin of error at about 2 standard errors, and report the estimate.

Solution

The point estimate of p, the share of all riders who are satisfied, is \displaystyle \hat{p} = \frac{384}{600} = 0.64. The standard error puts \hat{p} where the unknown p would go:

\begin{aligned} \text{SE}_{\hat{p}} &= \sqrt{\frac{0.64 \times 0.36}{600}} = \sqrt{\frac{0.2304}{600}} \\ &= \sqrt{0.000384} \approx 0.0196 \end{aligned}

The margin of error is twice that, \text{ME} \approx 2 \times 0.0196 \approx 0.039, so the report is 0.64 \pm 0.039:

\begin{aligned} 0.64 - 0.039 &= 0.601 \\ 0.64 + 0.039 &= 0.679 \end{aligned}

In context: the agency estimates that 64\% of its riders are satisfied, and the chance error of a random sample of this size is likely within about 3.9 percentage points, so the share is roughly 0.60 to 0.68. This is Figure 1.

Example 2

Minutes waited

Tahir asks a random sample of 64 riders how long they waited at their stop. The mean is \bar{x} = 8.6 minutes and s_x = 4.8 minutes (made-up data). Find the margin of error for the mean wait of all riders.

Solution

The point estimate of \mu, the mean wait of all riders, is \bar{x} = 8.6 minutes. Its standard error is

\text{SE}_{\bar{x}} = \frac{s_x}{\sqrt{n}} = \frac{4.8}{\sqrt{64}} = \frac{4.8}{8} = 0.6 \text{ minutes}

and the margin of error is 2 \times 0.6 = 1.2 minutes. The estimate is 8.6 \pm 1.2 minutes, from 7.4 to 9.8 minutes. Wait times are often skewed, with a few long waits, but 64 riders is enough for the sampling distribution of the mean to be close to normal, which is what the multiplier 2 relies on. The margin is in minutes, the units of the data.

Example 3

Halving the margin

The agency wants a margin half as large as 0.039. How many riders does that take, if \hat{p} stays near 0.64? And how many would give a margin of 0.025?

Solution

The margin shrinks like \displaystyle \frac{1}{\sqrt{n}}, so halving it takes 4 times the sample: 4 \times 600 = 2{,}400 riders. Checking,

\begin{aligned} 2\sqrt{\frac{0.2304}{2400}} &= 2\sqrt{0.000096} \\ &\approx 2 \times 0.0098 \approx 0.0196 \end{aligned}

half of 0.0392. For a margin of 0.025, set the formula equal to it and solve. Both sides are positive, so squaring is allowed:

\begin{aligned} 2\sqrt{\frac{0.2304}{n}} &= 0.025 \\ \frac{4 \times 0.2304}{n} &= 0.000625 \\ n &= \frac{0.9216}{0.000625} = 1474.56 \end{aligned}

A sample must be a whole number of riders, and rounding down would leave the margin slightly above 0.025, so the agency needs 1{,}475 riders. Figure 2 shows the curve: early riders cut the margin a lot, later ones much less.

Example 4

Same sample, different margins

Shabnam reads two other questions from the same 600 riders. On the first, 300 would recommend the buses to a friend; on the second, 54 lost an item on a bus in the past year. Find each margin of error and compare them with the satisfaction margin of 0.039.

Solution

For recommending, \displaystyle \hat{p} = \frac{300}{600} = 0.5:

\begin{aligned} \text{ME} &\approx 2\sqrt{\frac{0.5 \times 0.5}{600}} = 2\sqrt{0.000417} \\ &\approx 2 \times 0.0204 \approx 0.0408 \end{aligned}

For lost items, \displaystyle \hat{p} = \frac{54}{600} = 0.09:

\begin{aligned} \text{ME} &\approx 2\sqrt{\frac{0.09 \times 0.91}{600}} = 2\sqrt{0.0001365} \\ &\approx 2 \times 0.0117 \approx 0.0234 \end{aligned}

The sample size is the same for all three questions, yet the margins are 0.041, 0.039 and 0.023. The even split, 0.5, has the largest factor \hat{p}(1 - \hat{p}) = 0.25; 0.64 gives 0.2304, a little less; 0.09 gives only 0.0819. A survey that quotes one margin for all its questions needs a figure that covers every one of them, and the margin at 0.5, here about 0.041, does.

Example 5

Reading a published margin backwards

A news story reports that 58\% of riders in a poll favor adding late-night routes, “with a margin of error of plus or minus 3 percentage points.” Assuming a random sample and a multiplier of 2, about how many riders were polled?

Solution

Three percentage points is 0.03. If the margin was computed from the poll’s own \hat{p} = 0.58,

\begin{aligned} 2\sqrt{\frac{0.58 \times 0.42}{n}} &= 0.03 \\ \frac{4 \times 0.2436}{n} &= 0.0009 \\ n &\approx 1082.7 \end{aligned}

about 1{,}083 riders. If it was computed at \hat{p} = 0.5, the margin that covers any proportion, the bound from Results gives

n = \frac{1}{0.03^2} = \frac{1}{0.0009} \approx 1111.1

about 1{,}111 riders. Either way the poll asked roughly 1{,}100 people. A reported margin, read backwards, tells the size of the sample behind it.

Example 6

A wider margin for more samples

In a normal distribution about 99.7\% of values lie within 3 standard deviations of the center. Use a multiplier of 3 on Mehwish’s sample and compare with the 2-standard-error margin.

Solution

The standard error is still 0.0196; only the multiplier changes:

\text{ME} \approx 3 \times 0.0196 \approx 0.059

The estimate becomes 0.64 \pm 0.059, from 0.581 to 0.699. The range is half again as wide as 0.601 to 0.679. Covering more samples, about 99.7\% rather than about 95\%, costs width: the same data, with a stronger claim of coverage, give a less precise statement. The multipliers 2 and 3 are rounded; the exact ones come from the Critical value.

Example 7

What the margin leaves out

Mehwish learns more about her survey. The 600 answers came from 1{,}500 riders who were contacted, and the rest never replied. The riders were contacted by email, and only riders who had registered a fare card had an email on file. The question read “How satisfied are you with our improved service?” Explain what the margin of 0.039 covers, and find the margin if 6{,}000 answers had come in the same way.

Solution

The margin of error measures one thing: how much \hat{p} would vary by chance from one random sample to another. It assumes the 600 are a random sample of the population of interest. None of the three problems is chance:

Nonresponse. If satisfied riders were more willing to reply, the 600 lean satisfied however they were picked.
Undercoverage. Riders who pay cash had no chance of being asked.
Wording. “Our improved service” invites agreement, a response bias.

Each of these can push the estimate off in one direction, and no formula from the sample shows how far. With 6{,}000 answers gathered the same way,

\begin{aligned} 2\sqrt{\frac{0.2304}{6000}} &= 2\sqrt{0.0000384} \\ &\approx 0.012 \end{aligned}

The margin shrinks, but around a value that may be biased. A small margin is a statement about chance error, never a guarantee of accuracy.

Boundaries

The margin of error covers random sampling error only. Bias from undercoverage, nonresponse, the wording of a question, or a convenience sample is not in it, and a larger sample does not reduce that bias.
The formulas need a random sample of independent observations. Sampling without replacement is fine when the sample is under a tenth of the population (The ten percent condition). Within that condition the margin depends on the sample size, not on the size of the population.
The multiplier 2 for about 95\% relies on a sampling distribution close to normal. For a proportion that needs enough successes and failures in the sample; for a mean it needs a large enough sample or a population not strongly skewed. Small samples need other methods.
When \hat{p} is 0 or 1, the formula gives a margin of 0, though the estimate is not exact. The standard error fails at the ends.
A margin of error describes a method’s typical reach, not the distance for this sample. A given sample may be farther than the margin from the parameter; about 5 samples in 100 are.

Where this goes next

The report \hat{p} \pm \text{ME} is a confidence interval in all but name. The Critical value replaces the rounded multiplier 2 with the exact number for any chosen percentage, and the Confidence level and interpreting an interval says precisely what “95\%” claims about the method and what it does not claim about a single interval. Planning a study backwards from a desired margin, as in Examples 3 and 5, becomes choosing a sample size before any data are collected. And in every interval of this course, for proportions, means, and differences of two, the shape is the same: point estimate, plus or minus a multiplier times a standard error.

Exercises

Point estimates and margins

A random sample of 500 riders has 410 who say the buses are clean. Find \hat{p}, the standard error, the margin of error at about 2 standard errors, and the range it gives.
In a random sample of 250 riders, 95 use a monthly pass. Find the margin of error and the range.
A report gives the share of riders who use the agency’s app as 0.47 to 0.55. Find the point estimate, the margin of error, and the standard error the margin was built on.
A random sample of 80 riders has a mean trip length of 6.2 miles. Name the parameter and the point estimate, in context.

Margins for a mean

A random sample of 100 riders has a mean trip time of 22.5 minutes with s_x = 9 minutes. Find the margin of error and the range.
A random sample of 36 riders spends a mean of 18.40 dollars a week on fares, with s_x = 6.30 dollars. Find the margin of error and the range.
From a random sample of 49 riders, the mean wait is reported as 11.2 to 13.6 minutes using a multiplier of 2. Find \bar{x}, the margin of error, the standard error and s_x.

What moves the margin

Suppose Mehwish had asked only 100 riders and found the same \hat{p} = 0.64. Find the margin of error and compare it with 0.039.
With \hat{p} near 0.64, how many riders give a margin of error of 0.03?
The agency plans a survey on a new question and has no guess at the answer. How many riders guarantee a margin of error of 0.05?
For random samples of 900 riders, find the margin of error when \hat{p} is 0.5, 0.7 and 0.9.
Use a multiplier of 3 on the sample in Exercise 1. Find the margin and the range.
Trip times have s_x near 6 minutes. How many riders give a margin of error of 1 minute for the mean?

Reading published margins

A report says 61\% of riders support later service, plus or minus 4 percentage points. Find the implied sample size two ways: from 0.61, and from the bound at 0.5.
A survey claims a margin of plus or minus 2 percentage points for every question it asked. At least how many people were in the sample?
One poll asked 400 riders and another 1{,}600. Find the margin at \hat{p} = 0.5 for each and say how they compare.

Reasoning

Two cities, one with 40{,}000 riders and one with 4 million, each survey a random sample of 600 riders and find \hat{p} = 0.64. Explain why the margins of error are essentially the same.
A survey of riders who chose to answer a poll on the agency’s app reports a margin of error of 0.03. Explain what that margin does and does not account for.
Shabnam says the margin of 0.039 means that 3.9\% of the riders answered the question wrongly. Explain what it does measure.
Doubling the sample does not halve the margin. Find the factor by which it shrinks, and the margin for 1{,}200 riders with \hat{p} = 0.64.
Show that \hat{p}(1 - \hat{p}) is never more than 0.25, and explain why that makes \displaystyle \frac{1}{\sqrt{n}} a margin that covers every question in a survey.
Tahir wants to halve Mehwish’s margin of 0.0392 and, at the same time, use a multiplier of 3 instead of 2. Explain how many riders that takes, with \hat{p} near 0.64.

Answers

\displaystyle \hat{p} = \frac{410}{500} = 0.82; \displaystyle \text{SE} = \sqrt{0.82 \times 0.18 / 500}, which is \displaystyle \sqrt{0.0002952} \approx 0.0172; \text{ME} \approx 0.0344; range 0.786 to 0.854.
\displaystyle \hat{p} = \frac{95}{250} = 0.38; \displaystyle \text{SE} = \sqrt{0.38 \times 0.62 / 250}, which is \displaystyle \sqrt{0.0009424} \approx 0.0307; \text{ME} \approx 0.0614; range 0.319 to 0.441.
Point estimate \displaystyle \frac{0.47 + 0.55}{2} = 0.51; margin \displaystyle \frac{0.55 - 0.47}{2} = 0.04, half the width; standard error about \displaystyle \frac{0.04}{2} = 0.02.
The parameter is the mean trip length \mu of all the agency’s riders. The point estimate is the sample mean \bar{x} = 6.2 miles from the 80 riders.
\displaystyle \text{SE} = \frac{9}{\sqrt{100}} = 0.9 minutes; \text{ME} = 1.8 minutes; range 20.7 to 24.3 minutes.
\displaystyle \text{SE} = \frac{6.30}{6} = 1.05 dollars; \text{ME} = 2.10 dollars; range 16.30 to 20.50 dollars a week.
\bar{x} = 12.4 minutes, the midpoint; \text{ME} = 1.2 minutes; \text{SE} = 0.6 minutes; \displaystyle s_x = 0.6 \times \sqrt{49} = 4.2 minutes.
\displaystyle 2\sqrt{0.2304/100} is \displaystyle 2\sqrt{0.002304}, which is 2 \times 0.048 = 0.096, about 2.45 times 0.039: a sixth of the sample multiplies the margin by \displaystyle \sqrt{6} \approx 2.45.
\displaystyle n = \frac{4 \times 0.2304}{0.03^2} = \frac{0.9216}{0.0009} = 1024 riders.
\displaystyle n = \frac{1}{0.05^2} = 400 riders, from the bound at \hat{p} = 0.5.
At 0.5: \displaystyle 2\sqrt{0.25/900} \approx 0.0333. At 0.7: \displaystyle 2\sqrt{0.21/900} \approx 0.0306. At 0.9: \displaystyle 2\sqrt{0.09/900} = 0.02.
\text{ME} \approx 3 \times 0.0172 \approx 0.0515; range 0.768 to 0.872.
\displaystyle \frac{2 \times 6}{\sqrt{n}} = 1 gives \displaystyle \sqrt{n} = 12, so n = 144 riders.
From 0.61: \displaystyle n = \frac{4 \times 0.2379}{0.0016} = 594.75, about 595. From 0.5: \displaystyle n = \frac{1}{0.04^2} = 625.
A margin for every question must hold at \hat{p} = 0.5: \displaystyle n = \frac{1}{0.02^2} = 2500 people.
\displaystyle \frac{1}{\sqrt{400}} = 0.05 and \displaystyle \frac{1}{\sqrt{1600}} = 0.025. Four times the sample, half the margin.
The formula \displaystyle 2\sqrt{\hat{p}(1 - \hat{p})/n} has no population size in it. Both samples are far under a tenth of their populations, so the observations are close to independent, and the chance variation depends only on how many riders were asked.
It accounts for the chance variation of a random sample of that size. It does not account for the bias of a voluntary response sample: riders who choose to answer may differ from riders who do not, and no margin from the sample measures that.
It is not about wrong answers. It says that sample proportions from random samples of 600 riders are likely, in about 95\% of samples, to land within about 0.039 of the true share, through chance in which riders were picked.
The margin is divided by \displaystyle \sqrt{2}, so it shrinks by a factor of about 0.707. For 1{,}200 riders, \displaystyle 2\sqrt{0.2304/1200} \approx 0.0277.
\hat{p}(1 - \hat{p}) = 0.25 - (\hat{p} - 0.5)^2, and a square is never negative, so the product is at most 0.25. Then \displaystyle 2\sqrt{\hat{p}(1 - \hat{p})/n} \le 2\sqrt{0.25/n} = \frac{1}{\sqrt{n}} for every question, whatever its proportion.
The new margin must be \displaystyle \frac{0.0392}{2} = 0.0196 with multiplier 3, so the standard error must be \displaystyle \frac{0.0196}{3}, a third of the original 0.0196. That takes 9 times the sample, 5{,}400 riders: \displaystyle 3\sqrt{0.2304/5400} \approx 0.0196.