Statistics

Standard error

The spread of a statistic’s sampling distribution depends on the very parameter we are trying to estimate. How can one sample say how far off it might be?

You need

Bias and variability of an estimatorStatistics

Unlocks

Point estimates and the margin of errorStatistics Test statisticStatistics

Where you are 4 in Confidence intervals

Statistics7

Statistical inference8

Confidence intervals4

Confidence level and interpreting an interval Point estimates and the margin of error

Standard error

Critical value

Why this exists

A university genetics lab keeps a large culture of fruit flies, some with curly wings. Ioane wants the share of curly-winged flies in the whole culture, the parameter p. Counting every fly is out of the question, so he takes a random sample of 200 flies and finds 46 with curly wings (made-up data):

\hat{p} = \frac{46}{200} = 0.23

The estimate 0.23 is almost surely not exactly p. How far off might it be? Sampling variability gave the answer in principle: the spread of the sampling distribution of \hat{p}, the pile of sample proportions from all possible samples of 200. For random samples that spread has a formula,

\sigma_{\hat{p}} = \sqrt{\frac{p(1 - p)}{n}}

and there is the snag. The formula needs p, the number Ioane is trying to find. If he knew p he would not need the sample.

The way out is to put the estimate where the unknown parameter was. That plug-in number, computed from the one sample in hand, is the standard error.

Definition

The standard error of a statistic is an estimate, computed from the sample data, of the standard deviation of that statistic’s sampling distribution. It is written \text{SE} with the statistic as a subscript.

Two numbers must be kept apart:

	Uses	Can it be computed from the sample alone?
standard deviation of the sampling distribution, \sigma_{\hat{p}} or \sigma_{\bar{x}}	the parameter, p or \sigma	no
standard error, \text{SE}_{\hat{p}} or \text{SE}_{\bar{x}}	the statistic, \hat{p} or s_x	yes

For a sample proportion and a sample mean from a random sample of size n:

\text{SE}_{\hat{p}} = \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}

\text{SE}_{\bar{x}} = \frac{s_x}{\sqrt{n}}

where s_x is the sample standard deviation (dividing by n - 1). The standard error is in the units of the statistic: a proportion for \hat{p}, millimeters for a mean length in millimeters.

Some books call \sigma_{\hat{p}} a standard error too. Here the words are kept apart: the standard deviation of the sampling distribution uses the parameter, and the standard error is the estimate of it from the sample.

Intuition

The standard error answers “about how far does a statistic like this one typically land from the parameter?” For Ioane, a typical sample proportion from 200 flies lands about 0.03 from the culture’s true share. It does not say which way his landed, or how far; it describes the method’s scatter, the way the variability of an estimator did. That lesson asked two questions of an estimator: is its pile centered on the parameter, and how widely does it spread? The sample proportion from a random sample is unbiased, and the standard error is a number for the second question.

Why should plugging in \hat{p} be safe? Figure 1 draws \displaystyle \sqrt{\hat{p}(1 - \hat{p})/200} for every value \hat{p} could take.

Figure 1The standard error \displaystyle \sqrt{\hat{p}(1 - \hat{p})/200} for every possible sample proportion from a sample of 200 flies (made-up data). Near the middle the curve is nearly flat, so a sample proportion a little off from p still gives almost the right spread.

The curve is flat near the top. If the true p were 0.20 or 0.26 rather than 0.23, the spread would barely move, so being a little wrong about p costs almost nothing in the spread. Only near 0 or 1, where the curve drops steeply to 0, does a small error in \hat{p} change the standard error a lot.

Results

For a random sample of n independent observations:

\begin{aligned} \sigma_{\hat{p}} &= \sqrt{\frac{p(1 - p)}{n}} \quad \text{is estimated by} \\ \text{SE}_{\hat{p}} &= \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}} \end{aligned}

\begin{aligned} \sigma_{\bar{x}} &= \frac{\sigma}{\sqrt{n}} \quad \text{is estimated by} \\ \text{SE}_{\bar{x}} &= \frac{s_x}{\sqrt{n}} \end{aligned}

Both have \displaystyle \sqrt{n} underneath, so multiplying the sample size by 4 halves the standard error, and by 9 cuts it to a third.

For samples of size n, \text{SE}_{\hat{p}} is largest when \hat{p} = 0.5, where it equals \displaystyle \sqrt{0.25/n} = \frac{0.5}{\sqrt{n}}.

Every statistic used for inference has a standard error, not only these two. The slope of a least-squares line has one, printed as “SE Coef” in Reading regression output.

Why it is true

The spread of a proportion. Score each fly 1 if curly and 0 if not. One fly’s score has mean p and variance p \cdot 1^2 + (1 - p) \cdot 0^2 - p^2 = p(1 - p) (Variance and standard deviation of a random variable). The number of curly flies X is the sum of n independent scores, so its variance is np(1 - p), because variances add for independent variables. Dividing X by n divides the variance by n^2:

\text{Var}(\hat{p}) = \frac{np(1 - p)}{n^2} = \frac{p(1 - p)}{n}

and the standard deviation is its square root. The same argument for a mean of n independent values, each with variance \sigma^2, gives \displaystyle \text{Var}(\bar{x}) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}. The full treatment is in Sampling distribution of a sample proportion and Sampling distribution of a sample mean.

Why the plug-in is close. The sample proportion is unbiased for p, and its spread shrinks like \displaystyle \frac{1}{\sqrt{n}}, so for a large sample \hat{p} lies near p and \hat{p}(1 - \hat{p}) lies near p(1 - p). In the same way s_x lies near \sigma. The plug-in is not exact, though. Squaring the standard error and averaging over all samples,

\begin{aligned} \text{mean of } \hat{p}(1 - \hat{p}) &= p - \left(\text{Var}(\hat{p}) + p^2\right) \\ &= p(1 - p) - \frac{p(1 - p)}{n} \\ &= \frac{n - 1}{n}\, p(1 - p) \end{aligned}

using mean of \hat{p}^2 = variance + (mean)^2. So \text{SE}_{\hat{p}}^2 runs a little low, by the same factor \displaystyle \frac{n - 1}{n} met with the divide-by-n variance. For n = 200 the shortfall is half of one percent; for tiny samples it is large (Example 6).

Worked examples

Example 1

The curly-wing standard error

Find and interpret the standard error of Ioane’s sample proportion, 46 curly flies in a random sample of 200.

Solution

The sample proportion is \displaystyle \hat{p} = \frac{46}{200} = 0.23. The parameter p is unknown, so the spread is estimated with \hat{p} in its place:

\begin{aligned} \text{SE}_{\hat{p}} &= \sqrt{\frac{0.23 \times 0.77}{200}} = \sqrt{\frac{0.1771}{200}} \\ &= \sqrt{0.0008855} \approx 0.0298 \end{aligned}

Interpretation: sample proportions of curly-winged flies from random samples of 200 from this culture typically differ from the culture’s true proportion by about 0.03, or 3 percentage points.

Example 2

A claimed value: standard deviation, not standard error

The lab’s stock records say the culture is 25\% curly. Pita asks how much sample proportions from samples of 200 would vary if the records are right, and how far Ioane’s 0.23 sits from 0.25 in those units.

Solution

Now a value of the parameter is given, so the spread uses it. This is the standard deviation of the sampling distribution, not a standard error:

\sigma_{\hat{p}} = \sqrt{\frac{0.25 \times 0.75}{200}} = \sqrt{0.0009375} \approx 0.0306

It is close to the standard error 0.0298, because 0.25 is close to 0.23 on the flat part of Figure 1. Ioane’s result is \displaystyle \frac{0.23 - 0.25}{0.0306} \approx -0.65, about 0.65 standard deviations below what the records predict, a z-score for a sample proportion. A gap that small is ordinary sampling variability. Measuring a result in such units against a claimed value is what a Test statistic does.

Example 3

The standard error of a mean wing length

Vailea measures the wing length of a random sample of 36 flies from the culture. The mean is \bar{x} = 2.14 millimeters and the sample standard deviation is s_x = 0.15 millimeters (made-up data). Find the standard error of the mean. The lab’s long records give \sigma = 0.14 millimeters for wing length; find \sigma_{\bar{x}} too, and say which number Vailea can compute from her sample alone.

Solution

The standard error puts the sample’s s_x where \sigma would go:

\begin{aligned} \text{SE}_{\bar{x}} &= \frac{s_x}{\sqrt{n}} = \frac{0.15}{\sqrt{36}} \\ &= \frac{0.15}{6} = 0.025 \text{ mm} \end{aligned}

Means of 36 wing lengths typically land about 0.025 millimeters from the culture’s true mean wing length. If the records’ \sigma is taken as the population value,

\sigma_{\bar{x}} = \frac{0.14}{6} \approx 0.0233 \text{ mm}

Only the standard error comes from her sample alone; \sigma_{\bar{x}} depends on trusting the records’ value of a parameter.

Example 4

More flies

How large a sample would cut Ioane’s standard error in half? About how many flies would bring it to 0.01, if the proportion stays near 0.23?

Solution

The sample size sits under a square root, so halving the standard error takes 4 times as many flies, 800. With the same \hat{p},

\sqrt{\frac{0.1771}{800}} \approx 0.0149

half of 0.0298. For a standard error of 0.01, set the formula equal and solve:

\begin{aligned} \sqrt{\frac{0.1771}{n}} &= 0.01 \\ \frac{0.1771}{n} &= 0.0001 \\ n &= 1771 \end{aligned}

Squaring is allowed because both sides are positive. About 1{,}771 flies would be needed, nearly nine times the original sample, for a standard error a third the size.

Example 5

A second sample

Pita takes her own random sample of 200 flies from the same culture and finds 52 curly. Find her standard error and compare her results with Ioane’s.

Solution

Her sample proportion is \displaystyle \hat{p} = \frac{52}{200} = 0.26, and

\text{SE}_{\hat{p}} = \sqrt{\frac{0.26 \times 0.74}{200}} = \sqrt{0.000962} \approx 0.0310

The two estimates, 0.23 and 0.26, differ by 0.03, about one standard error: just the size of disagreement the standard error leads one to expect between two samples. The standard errors themselves, 0.0298 and 0.0310, nearly agree. The standard error is a statistic too and changes from sample to sample, but slowly, because the curve in Figure 1 is flat there.

Example 6

When the plug-in fails: a sample of two

To see the plug-in at its worst, take samples of only n = 2 flies, drawn independently, from a culture with p = 0.5. Find \sigma_{\hat{p}}, and the mean of \text{SE}_{\hat{p}} over all samples.

Solution

The true spread is

\sigma_{\hat{p}} = \sqrt{\frac{0.5 \times 0.5}{2}} = \sqrt{0.125} \approx 0.354

The samples are both curly, one of each, or neither, with probabilities 0.25, 0.5 and 0.25, giving \hat{p} = 1, 0.5 and 0. When \hat{p} is 0 or 1, \hat{p}(1 - \hat{p}) = 0 and the standard error is 0. When \hat{p} = 0.5 it is \displaystyle \sqrt{0.125} \approx 0.354. So

\text{mean of SE}_{\hat{p}} = 0.5 \times 0.354 \approx 0.177

half the true spread. The squared standard error averages 0.5 \times 0.125 = 0.0625, which is \displaystyle \frac{n - 1}{n} = \frac{1}{2} of \sigma_{\hat{p}}^2, as Why it is true predicts. With 2 flies the standard error is useless; with 200 the same factor is 0.995.

Boundaries

The formulas assume a random sample with independent observations. Sampling without replacement is close enough when the sample is under a tenth of the population (The ten percent condition); a large fly culture easily qualifies.
The standard error measures chance variation only. A sample collected in a way that favors one kind of fly is off by an amount no standard error shows, and a larger sample shrinks the standard error around the wrong value.
\text{SE}_{\hat{p}} is 0 when \hat{p} is 0 or 1, even though the estimate is not exact. Near those ends the plug-in breaks down.
The plug-in needs a reasonably large sample: for small n the standard error runs low on average (Example 6), and s_x from a few values can be far from \sigma.
The standard error gives a typical size of the error, not its shape. Turning it into a statement about how often the estimate lands within some distance needs a normal shape from the The central limit theorem, and that is the work of the following lessons, not of the standard error itself.

Where this goes next

The standard error is the building block of every interval in this unit. Point estimates and the margin of error multiplies it to say how far the estimate might reasonably sit from the truth, the Critical value is that multiplier, and the Confidence level and interpreting an interval says what share of samples such a method captures. (The rough margin of twice the spread from Simulating a sampling distribution and a margin of error is the same idea, built before the standard error existed.) In hypothesis tests the standard error appears again, as the unit a Test statistic counts in.

Exercises

Standard error of a proportion

A random sample of 150 flies from a second culture has 63 curly. Find and interpret the standard error of the sample proportion.
A random sample of 400 flies has 90 curly. Find \hat{p} and its standard error.
A random sample of 80 flies has 12 curly. Find \hat{p} and its standard error.

Standard error of a mean

A random sample of 49 flies has mean wing length 2.08 millimeters and s_x = 0.21 millimeters. Find and interpret \text{SE}_{\bar{x}}.
Vailea measures the wings of a random sample of 8 flies, in millimeters: 2.10, 2.25, 1.98, 2.31, 2.17, 2.05, 2.22, 2.12. Find \bar{x}, s_x and the standard error of the mean.
A random sample of 64 females laid a mean of 41.5 eggs each in one day, with s_x = 12 eggs. Find the standard error of the mean.

Parameter or statistic

A supplier claims the culture is 30\% curly. Find \sigma_{\hat{p}} for samples of 200 if the claim is true, and how many of these standard deviations Ioane’s 0.23 sits from 0.30.
The lab’s records give \sigma = 0.16 millimeters for wing length. A random sample of 64 flies has s_x = 0.18 millimeters. Find \sigma_{\bar{x}} and \text{SE}_{\bar{x}}.
Say whether each is a standard deviation of a sampling distribution or a standard error, and why. (a) \displaystyle \sqrt{0.5 \times 0.5 / 100} for the share of heads in 100 tosses of a fair coin. (b) \displaystyle s_x / \sqrt{30} from 30 measured wings. (c) \displaystyle \sqrt{0.26 \times 0.74 / 200} from Pita’s sample. (d) \displaystyle \sigma / \sqrt{n} with \sigma known.

Sample size

With \hat{p} near 0.23, about how many flies give a standard error of 0.02?
Ioane’s standard error was 0.0298 for 200 flies. Without the full formula, estimate it for 1{,}800 flies with the same \hat{p}, then check with the formula.
Wing lengths have s_x near 0.15 millimeters. How many flies give \text{SE}_{\bar{x}} = 0.01 millimeters?
For samples of 200, which sample proportion gives the largest standard error, and what is it?

How well the plug-in works

Samples of 2 flies are drawn independently from a culture with p = 0.2. Find \sigma_{\hat{p}} and the mean of \text{SE}_{\hat{p}} over all samples.
For samples of 50, the squared standard error averages what fraction of \sigma_{\hat{p}}^2? By what percent does it run low?

Reasoning

Explain why the standard error uses \hat{p} and what would be needed to compute \sigma_{\hat{p}} instead.
Pita says the standard error of 0.0298 means that 2.98\% of the flies were miscounted. Explain what it does measure.
A random sample of 50 flies from a third culture has no curly flies. Explain why the standard error of 0 does not mean the estimate \hat{p} = 0 is exact.
Suppose (a made-up premise) that curly-winged flies climb less, and Ioane collects only flies that climb into a tube at the top of the culture. Explain what the standard error will and will not tell him.
Explain from the formula why four times as many flies halves the standard error.

Answers

\displaystyle \hat{p} = \frac{63}{150} = 0.42; \displaystyle \text{SE}_{\hat{p}} = \sqrt{0.42 \times 0.58 / 150}, which is \displaystyle \sqrt{0.001624} \approx 0.0403. Sample proportions from 150 flies typically land about 0.04 from that culture’s true share.
\hat{p} = 0.225; \displaystyle \text{SE}_{\hat{p}} = \sqrt{0.225 \times 0.775 / 400}, which is \displaystyle \sqrt{0.000435938} \approx 0.0209.
\hat{p} = 0.15; \displaystyle \text{SE}_{\hat{p}} = \sqrt{0.15 \times 0.85 / 80}, which is \displaystyle \sqrt{0.00159375} \approx 0.0399.
\displaystyle \frac{0.21}{\sqrt{49}} = \frac{0.21}{7} = 0.03 mm. Means of 49 wing lengths typically land about 0.03 millimeters from the true mean.
Sum 17.20, \bar{x} = 2.15 mm. Squared deviations add to 0.0832, \displaystyle s_x = \sqrt{0.0832/7} \approx 0.109 mm. \displaystyle \text{SE}_{\bar{x}} = \frac{0.109}{\sqrt{8}} \approx 0.0385 mm.
\displaystyle \frac{12}{\sqrt{64}} = \frac{12}{8} = 1.5 eggs.
\displaystyle \sigma_{\hat{p}} = \sqrt{0.30 \times 0.70 / 200}, which is \displaystyle \sqrt{0.00105} \approx 0.0324. \displaystyle \frac{0.23 - 0.30}{0.0324} \approx -2.16: about 2.16 standard deviations below the claim, an unusually large gap if the claim is true.
\displaystyle \sigma_{\bar{x}} = \frac{0.16}{8} = 0.02 mm; \displaystyle \text{SE}_{\bar{x}} = \frac{0.18}{8} = 0.0225 mm.
(a) Standard deviation: p = 0.5 is known for a fair coin. (b) Standard error: s_x is a statistic. (c) Standard error: 0.26 is a sample proportion. (d) Standard deviation: it uses the parameter \sigma.
\displaystyle n = \frac{0.1771}{0.02^2} = 442.75, so about 443 flies.
9 times the flies divides the standard error by 3: about 0.0099. Formula: \displaystyle \sqrt{0.1771/1800} \approx 0.0099.
\displaystyle \frac{0.15}{\sqrt{n}} = 0.01 gives \displaystyle \sqrt{n} = 15, n = 225 flies.
\hat{p} = 0.5; \displaystyle \sqrt{0.25/200} \approx 0.0354, the peak in Figure 1.
\displaystyle \sigma_{\hat{p}} = \sqrt{0.2 \times 0.8 / 2}, which is \displaystyle \sqrt{0.08} \approx 0.283. One curly fly, \hat{p} = 0.5, has probability 2(0.2)(0.8) = 0.32 and \displaystyle \text{SE} = \sqrt{0.125} \approx 0.354; otherwise \text{SE} = 0. Mean 0.32 \times 0.354 \approx 0.113, well under 0.283.
\displaystyle \frac{n - 1}{n} = \frac{49}{50} = 0.98, so on average the squared standard error runs 2\% below \sigma_{\hat{p}}^2.
The spread of \hat{p} is \displaystyle \sqrt{p(1 - p)/n}, and p is unknown; that is why the sample was taken. The standard error puts \hat{p} in its place. Computing \sigma_{\hat{p}} would need the true p, or a claimed value of it.
It says nothing about counting mistakes. It estimates how far sample proportions from random samples of 200 typically land from the culture’s true proportion, about 0.03, from chance in which flies were picked.
With \hat{p} = 0 the formula’s factor \hat{p}(1 - \hat{p}) is 0, but the culture may still hold a small share of curly flies that the sample missed. The plug-in fails at the ends of the curve in Figure 1.
The standard error still gives the chance scatter of proportions from samples collected this way. It cannot show the bias: if curly flies stay below, every sample runs low, and the standard error does not include that error.
n sits under a square root in the denominator. Replacing n by 4n multiplies the denominator by \displaystyle \sqrt{4} = 2, which halves the standard error.