The t distribution and degrees of freedom
A sample mean is judged by how many standard deviations of the mean it sits from a claimed value, but that standard deviation needs the population’s spread, which nobody knows. Put the sample’s own standard deviation in its place, and what distribution does the result follow?
Where you are 7 in Inference for means
Why this exists
Morag runs a sugarhouse in Vermont, where maple sap is boiled down to syrup. Sweeter sap needs less boiling for each gallon of syrup, so she wants to know the mean sugar content of the sap across all her trees. For planning she has assumed 2.0 percent. Catriona measures the sap of a random sample of 8 trees (made-up data, in percent sugar):
The sample mean is \bar{x} = 2.125. In Sampling distribution of a sample mean, a sample mean from a normal population was judged by standardizing it, \displaystyle z = \dfrac{\bar{x} - \mu}{\sigma/\sqrt{n}}, and that z follows the The standard normal distribution; this is how a light sample of coffee bags was shown to be unusual. Here the population standard deviation \sigma of the sugar content is not known. The best Morag has is the sample’s own standard deviation, s \approx 0.301, which gives the Standard error \displaystyle \dfrac{s}{\sqrt{n}} \approx 0.106.
So the ratio she can compute is \displaystyle \dfrac{\bar{x} - \mu}{s/\sqrt{n}}. Is it standard normal? It is not, and the reason is that its denominator is now a statistic: it changes from sample to sample along with the numerator. This lesson describes the distribution it does follow.
Definition
For a random sample of size n from a population with mean \mu, the t statistic is
It measures how many standard errors the sample mean lies from \mu.
When the population is normal, the values of t over all possible samples of size n follow the t distribution with n - 1 degrees of freedom. The t distributions are a family of curves, one for each positive number of degrees of freedom. Every one of them is
- centered at 0 and symmetric about 0,
- bell-shaped, with more spread than the standard normal curve and more area far out in its tails,
- closer to the standard normal curve the more degrees of freedom it has.
The degrees of freedom of a sample standard deviation is \text{df} = n - 1, the number of deviations x - \bar{x} that are free to take any value. The n deviations always add to 0, so once n - 1 of them are known the last is forced. It is the same n - 1 that divides the sum of squared deviations in s, for the reason given in Bias and variability of an estimator.
When the population standard deviation \sigma is known, there is no estimate in the denominator, \displaystyle z = \dfrac{\bar{x} - \mu}{\sigma/\sqrt{n}} is standard normal, and the z procedures of Critical value apply. That case is rare, and Day 2 says when it happens.
Intuition
The t statistic is the z statistic with \sigma swapped for s, and the swap can be written as one factor:
In a sample whose s comes in above \sigma, the factor is below 1 and pulls t in toward 0. In a sample whose s comes in below \sigma, the factor is above 1 and pushes t out. The two do not cancel. A small s can make the factor large (s at half of \sigma doubles t), and in small samples s falls below \sigma more often than above it: for samples of 4 from a normal population, s < \sigma in about 61\% of samples (a fact stated here without proof). Those samples, which make the spread look smaller than it is, are the ones that throw t far out, and they fill the tails in Figure 1.
As the sample grows, s settles close to \sigma, the factor \displaystyle \frac{\sigma}{s} settles close to 1, and t behaves like z. That is why the curves in Figure 1 close in on the standard normal curve as the degrees of freedom grow.
Results
Day 1: the t statistic and its distribution
The t statistic for one sample. For a random sample of size n from a normal population with mean \mu,
follows the t distribution with \text{df} = n - 1.
Shape. Each t distribution is centered at 0 and symmetric about 0. For \text{df} > 2 its standard deviation is \displaystyle \sqrt{\dfrac{\text{df}}{\text{df} - 2}}, which is always above 1 and falls toward 1 as df grows: about 1.183 for \text{df} = 7 and about 1.035 for \text{df} = 30.
Heavier tails. The area beyond 2 shrinks toward the standard normal value as df grows:
| curve | area to the right of 2 |
|---|---|
| t, \text{df} = 3 | 0.0697 |
| t, \text{df} = 7 | 0.0428 |
| t, \text{df} = 30 | 0.0273 |
| standard normal | 0.0228 |
Day 2: critical values and tail areas
Critical value. For a confidence level C, the critical value t^* for a given df is the positive number with central area C between -t^* and t^*. As with z^*, it is the point with area \dfrac{1 + C}{2} to its left. It is larger than z^* for every df, and closes in on z^* as df grows:
| \text{df} | 90\% | 95\% | 99\% |
|---|---|---|---|
| 3 | 2.353 | 3.182 | 5.841 |
| 4 | 2.132 | 2.776 | 4.604 |
| 7 | 1.895 | 2.365 | 3.499 |
| 10 | 1.812 | 2.228 | 3.169 |
| 20 | 1.725 | 2.086 | 2.845 |
| 30 | 1.697 | 2.042 | 2.750 |
| 60 | 1.671 | 2.000 | 2.660 |
| 1{,}000 | 1.646 | 1.962 | 2.581 |
| z^* | 1.645 | 1.960 | 2.576 |
The columns can equally be read as one-tail areas: 0.05, 0.025 and 0.005 to the right of the listed value.
Technology. On a graphing calculator, `invT(area to the left, df)` returns a t value and `tcdf(lower, upper, df)` returns the area between two values (a very large number such as 10^{99} stands in for “no upper end”). In a spreadsheet, `T.INV(area to the left, df)` returns a t value and `T.DIST.RT(t, df)` returns the area to the right of t.
When \sigma is known. For a random sample from a normal population whose standard deviation \sigma is known, \displaystyle z = \dfrac{\bar{x} - \mu}{\sigma/\sqrt{n}} is exactly standard normal, and an interval for \mu is \displaystyle \bar{x} \pm z^* \dfrac{\sigma}{\sqrt{n}}.
Why it is true
Why the tails are heavier. From \displaystyle t = z \cdot \frac{\sigma}{s}, the size of t is larger than the size of z exactly when s < \sigma. So a value of t beyond a cut such as \pm 1.96 can come from a large z, as before, or from an ordinary z divided by an s that came in low. The second route does not exist for z, and it adds area to the tails. The exact curve was worked out by William Gosset, a brewery chemist who wrote under the pen name Student, which is why the distribution is often called Student’s t.
Why n - 1. The deviations from the sample mean always add to zero:
So s is built from only n - 1 freely varying deviations. For a normal population the distribution of s, and so of t, depends on the sample size only through this count, which is why the t curve is labeled by n - 1 rather than by n.
Why the center is 0 and the curve symmetric. For a normal population, \bar{x} - \mu is as likely to be negative as positive by the same amount, and s is the same whichever sign it takes (for a normal population \bar{x} and s are in fact independent, a property stated here without proof). Flipping the sign of the numerator leaves the denominator alone, so t and -t have the same distribution.
Why the curves approach the normal. For large n, s is close to \sigma in almost every sample, so \displaystyle \frac{\sigma}{s} is close to 1 and t is close to z.
Worked examples
Day 1: the t statistic for eight trees
Find s, the standard error, the t statistic for Catriona’s 8 trees against Morag’s planning value \mu = 2.0, and its degrees of freedom.
SolutionThe mean is \displaystyle \bar{x} = \frac{17.0}{8} = 2.125. The deviations x - \bar{x} are
and their squares add to 0.635. Dividing by n - 1 = 7 and taking the square root,
The t statistic is
with \text{df} = 8 - 1 = 7. The sample mean lies about 1.17 standard errors above 2.0 percent. Whether that is unusual is judged against the t distribution with 7 degrees of freedom, not against the standard normal curve; that judgment is the work of One sample test for a mean.
Day 1: when s comes in low
Owain builds a simulated orchard whose sugar content is normal with \mu = 2.0 and \sigma = 0.5, so he knows both. One sample of 4 trees gives \bar{x} = 2.3 and s = 0.2; another gives \bar{x} = 2.3 and s = 0.8. Find z and t for each.
SolutionBoth samples have the same mean, and z uses the true \sigma, so both have
The t statistics differ, because each uses its own s:
Each matches \displaystyle t = z \cdot \frac{\sigma}{s}: 1.2 \times 2.5 = 3.0 and 1.2 \times 0.625 = 0.75. A z of 1.2 is unremarkable, yet the first sample’s spread came in at less than half of \sigma, and its t of 3.0 lands far out in a tail. That is how the t distribution comes to have more area out there than z.
Day 1: counting the degrees of freedom
Catriona reads out seven of the eight deviations from the first example: 0.275, -0.225, 0.075, -0.425, 0.475, -0.125, 0.175. What must the eighth be? Give the degrees of freedom for samples of 5 trees and of 25 trees.
SolutionThe seven add to 0.225. All eight must add to 0, so the eighth deviation is forced to be -0.225, which is what the first example found. Only seven deviations were free; the eighth carried no new information about spread. That is the meaning of \text{df} = 7.
The same count gives \text{df} = 5 - 1 = 4 for a sample of 5 trees and \text{df} = 25 - 1 = 24 for a sample of 25.
Day 1: a simulation of the tails
Owain draws 10{,}000 random samples of 4 trees from his simulated orchard (\mu = 2.0, \sigma = 0.5), and for each computes z with the true \sigma and t with the sample’s s. Then he repeats with samples of 5. His counts of statistics beyond \pm 1.96:
| sample size | |z| > 1.96 | |t| > 1.96 |
|---|---|---|
| 4 | 512 | 1{,}477 |
| 5 | 519 | 1{,}218 |
What do the counts show, and what does the t distribution predict?
SolutionAs proportions of 10{,}000, the z counts are 0.0512 and 0.0519, close to the 0.05 that the standard normal curve puts beyond \pm 1.96. The t counts are 0.1477 and 0.1218, between two and three times as large. Using \pm 1.96 with the sample’s own s would mark about one sample in seven or eight as unusual when the intended rate is one in twenty.
The t distribution predicts these rates. Beyond \pm 1.96 the t curve with \text{df} = 3 has area about 0.145, and with \text{df} = 4 about 0.122, matching the simulation up to chance. The rate falls from n = 4 to n = 5 because s is a steadier estimate of \sigma in the larger sample. Another run of the simulation would give slightly different counts.
Day 2: the critical value for 95% with 7 degrees of freedom
Find t^* for 95\% confidence with \text{df} = 7 from the table and from technology, and compare it with z^*.
SolutionThe central 0.95 leaves 0.025 in each tail, so t^* has 0.975 to its left. In the table, row 7 and the 95\% column give t^* = 2.365; `invT(0.975, 7)` or `T.INV(0.975, 7)` gives 2.3646.
The standard normal value is z^* = 1.960, and
so a margin of error built with t^* is about 21\% wider than one built with z^* from the same standard error. With Morag’s data the interval in One sample interval for a mean and sample size will read
The extra width is the price of estimating \sigma from 8 trees.
Day 2: a tail area, by table and by technology
Find the area to the right of t = 2 under the t curve with \text{df} = 7, first bracketed from the table and then exactly, and compare it with the standard normal area to the right of 2.
SolutionIn row 7 of a t table, 1.895 has 0.05 to its right and 2.365 has 0.025 to its right. Since 2 lies between them, the area to its right lies between 0.025 and 0.05. Technology narrows it: `tcdf(2, 10^99, 7)` or `T.DIST.RT(2, 7)` gives about 0.043.
The standard normal area to the right of 2 is about 0.023. The t area is nearly twice as large, the heavier tail in one number. Both tails together give about 2(0.0428) \approx 0.086 under the t curve, against 0.046 under the normal curve.
Day 2: when the spread is known
Morag’s refractometer, which reads sugar content, has been checked against a standard solution for years, and its readings scatter normally about the true value with standard deviation \sigma = 0.05 percent sugar, the same from batch to batch. Catriona takes 4 readings of one batch of syrup and gets a mean of 66.90 percent. Which distribution applies, and what is the 95\% interval for the batch’s true sugar content?
SolutionThe spread here belongs to the instrument, not to the batch: it was measured over a long record of readings, and it does not change when the batch does, while the mean does. So \sigma is known before the sample is taken, the standardized mean \displaystyle z = \frac{\bar{x} - \mu}{\sigma/\sqrt{n}} is exactly standard normal, and z^* = 1.960 applies:
so the batch is between about 66.851 and 66.949 percent sugar, with 95\% confidence. Had Catriona used only the s of her 4 readings, the multiplier would have been t^* = 3.182 with \text{df} = 3. The sap of the trees is different: no long record fixes the spread of sugar content across trees, so that \sigma is estimated from each sample, and the t distribution is the one to use.
Boundaries
- The t statistic follows a t distribution exactly only when the population is normal and the sample is random. For other populations it is approximately t, more closely as n grows, which is the subject of Conditions for one mean and, behind it, the The central limit theorem.
- \text{df} = n - 1 is the rule for one sample. Other procedures, such as those comparing two means, have their own degrees of freedom.
- A t statistic needs n \ge 2. A single value has no deviation free to vary, so s and the t statistic are undefined.
- For \text{df} = 1 and \text{df} = 2 the standard deviation formula does not apply: the tails are so heavy that the spread is not finite. For \text{df} = 1 even the mean does not exist, though the curve is still symmetric about 0.
- The z procedures need a \sigma known from outside the sample. Putting s in the place of \sigma and still using z^* gives intervals that are too narrow, worst for small samples.
- A table lists only some df. For a df not listed, using the next smaller df gives a slightly larger t^*, which errs toward a wider interval; technology gives the exact value.
Where this goes next
The t distribution plays for means the part the standard normal curve played for proportions. The One sample interval for a mean and sample size is \displaystyle \bar{x} \pm t^* \dfrac{s}{\sqrt{n}}, with the critical value read exactly as on Day 2, and the One sample test for a mean uses t as its Test statistic and a t tail area as its P-value. Both rest on the Conditions for one mean, which say when the t curve is a fair model. Later procedures for paired data and for two samples keep the same t family with their own degrees of freedom.
Exercises
Use technology for t values and areas, to three decimal places, unless a part says otherwise. All data are made up.
Day 1: the t statistic and degrees of freedom
- A random sample of 12 trees has \bar{x} = 2.31 and s = 0.42 percent sugar. Find the t statistic against \mu = 2.0 and its degrees of freedom.
- A random sample of 5 trees from a younger stand has \bar{x} = 1.84 and s = 0.30. Find the t statistic against \mu = 2.0 and its degrees of freedom.
- Give the degrees of freedom of s for samples of 15, 2 and 40 trees.
- Catriona samples 6 trees on a north slope: 1.8, 2.5, 2.2, 1.9, 2.4, 2.6. Find \bar{x}, s, the standard error, and the t statistic against \mu = 2.0.
- In Owain’s simulated orchard (\mu = 2.0, \sigma = 0.5), a sample of 16 trees has \bar{x} = 2.25 and s = 0.3. Find z and t, and check that \displaystyle t = z \cdot \frac{\sigma}{s}.
- A sample of 5 has four deviations from its mean of 0.3, -0.1, 0.4 and -0.2. Find the fifth.
- Order these three areas from smallest to largest without computing, then compute them: the area to the right of 3 under the standard normal curve, under the t curve with \text{df} = 4, and under the t curve with \text{df} = 20.
Day 2: critical values and tail areas
- Find t^* for 90\% confidence with \text{df} = 11.
- Find t^* for 99\% confidence with \text{df} = 4.
- Find t^* for 95\% confidence for a sample of 30 trees, and compare it with z^*.
- Find the area to the right of t = 1.5 with \text{df} = 9.
- Find the area to the left of t = -2.8 with \text{df} = 5.
- Find the total area beyond \pm 2.2 with \text{df} = 14.
- Using only the table values 2.228 (one tail 0.025) and 2.764 (one tail 0.01) for \text{df} = 10, bracket the area to the right of t = 2.6. Then find it exactly.
- Find the t value with area 0.05 to its right for \text{df} = 20.
- For 95\% confidence, find \displaystyle \frac{t^*}{z^*} for a sample of 4 and for a sample of 31. What does each ratio say about the width of an interval?
- Morag’s refractometer reads with known \sigma = 0.04 percent on a second batch. Five readings average 67.20 percent. Give the 95\% interval for that batch’s true sugar content.
- Say whether z or t applies, and with what df if t: (a) sugar content of sap from 10 trees; (b) 6 refractometer readings with the instrument’s long-run \sigma known; (c) sugar content of sap from 50 trees.
Reasoning
- Explain, using \displaystyle t = z \cdot \frac{\sigma}{s}, why the t distribution has heavier tails than the standard normal.
- Explain why the degrees of freedom of s is n - 1 and not n.
- Owain says that with 200 trees it makes no difference whether t^* or z^* is used for 95\%. Find t^* for that sample and say how far he is right.
- Why can a sample of 1 tree not give a t statistic?
- Owain repeats his simulation with samples of 30 trees. About what share of t statistics should fall beyond \pm 1.96?
- Catriona says the t curve with \text{df} = 7 is just a normal curve with standard deviation 1.183. Compare the area to the right of 2.365 under each, and say what the comparison shows.
Answers
- \displaystyle t = \frac{2.31 - 2.0}{0.42/\sqrt{12}} \approx 2.557, with \text{df} = 11.
- \displaystyle t = \frac{1.84 - 2.0}{0.30/\sqrt{5}} \approx -1.193, with \text{df} = 4. The negative sign says the sample mean is below 2.0.
- 14, 1 and 39.
- \displaystyle \bar{x} = \frac{13.4}{6} \approx 2.233; s \approx 0.327; \displaystyle \text{SE} = \frac{0.327}{\sqrt{6}} \approx 0.133; \displaystyle t \approx \frac{0.233}{0.133} = 1.75, with \text{df} = 5.
- \displaystyle z = \frac{0.25}{0.5/4} = 2.0 and \displaystyle t = \frac{0.25}{0.3/4} \approx 3.333. Check: \displaystyle 2.0 \times \frac{0.5}{0.3} \approx 3.333.
- The five add to 0, and the four add to 0.4, so the fifth is -0.4.
- Normal smallest, then \text{df} = 20, then \text{df} = 4, since fewer df means heavier tails. The areas are about 0.0013, 0.0035 and 0.0200.
- Area 0.95 to the left: t^* \approx 1.796.
- Area 0.995 to the left: t^* \approx 4.604.
- \text{df} = 29 and area 0.975 to the left: t^* \approx 2.045, against z^* = 1.960, so about 4\% larger.
- About 0.084.
- About 0.019.
- 2 \times 0.0226 \approx 0.045.
- 2.6 lies between 2.228 and 2.764, so the area is between 0.01 and 0.025. Exactly, about 0.013.
- Area 0.95 to the left: t \approx 1.725.
- \displaystyle \frac{3.182}{1.960} \approx 1.62 for \text{df} = 3 and \displaystyle \frac{2.042}{1.960} \approx 1.04 for \text{df} = 30. From the same standard error, the interval for 4 trees is about 62\% wider than a z interval would be; for 31 trees only about 4\% wider.
- \displaystyle 67.20 \pm 1.96 \cdot \frac{0.04}{\sqrt{5}} = 67.20 \pm 0.035, so about 67.165 to 67.235 percent.
- (a) t, \text{df} = 9, since \sigma for the trees is estimated by s. (b) z, since \sigma is known from the instrument’s long record. (c) t, \text{df} = 49; the t and z values are close at this size, but \sigma is still estimated.
- |t| > |z| exactly when s < \sigma. A sample with an ordinary z but a low s gives a large t, a second route into the tails that z does not have. In small samples s is below \sigma more often than above, and an s near 0 multiplies z by a factor with no upper limit, so area moves from the center out to the tails.
- The n deviations from \bar{x} add to 0, so only n - 1 are free and the last is fixed by the rest. The spread information in s comes from those n - 1, and the shape of the t curve depends on the sample only through that count.
- t^* \approx 1.972 with \text{df} = 199, against 1.960: a margin less than 1\% wider. He is right that the difference is small, but \sigma is still estimated, so t^* is the correct value and costs nothing with technology.
- With one value, the only deviation is x - \bar{x} = 0; there are 1 - 1 = 0 degrees of freedom, s would divide by 0, and so neither s nor t exists.
- With \text{df} = 29, the area beyond \pm 1.96 is about 0.060, so about 6\%, close to the 5\% for z.
- Under the t curve the area to the right of 2.365 is 0.025. Under a normal curve with standard deviation 1.183 it is the standard normal area beyond \displaystyle \frac{2.365}{1.183} \approx 1.999, about 0.023. The spreads match but the areas do not: the t curve is a different shape, not just a wider normal, and keeps more of its area far out.