Goodness of fit test
A genetics class crosses pea plants and predicts that the offspring seeds come in a 9 to 3 to 3 to 1 ratio of four types. When the counts come back uneven, how do you decide whether they still fit the prediction, and if they do not, which seed types are off and in which direction?
Where you are 3 in Chi-square tests
When you reach for it
Solomiia, Yaryna and Ostap are in a high school genetics class in Ukraine. They cross pea plants that each carry one dominant and one recessive version of two genes: seed shape (round over wrinkled) and seed color (yellow over green). Mendel’s model predicts that the offspring fall into four seed types in the ratio 9:3:3:1. The class raises 320 offspring and sorts them by seed type (made-up data):
| seed type | predicted share | observed |
|---|---|---|
| round yellow | \tfrac{9}{16} | 190 |
| round green | \tfrac{3}{16} | 52 |
| wrinkled yellow | \tfrac{3}{16} | 60 |
| wrinkled green | \tfrac{1}{16} | 18 |
| total | 1 | 320 |
Expected counts and the chi-square statistic already gives the expected counts E = np, the statistic \displaystyle \chi^2 = \sum \frac{(O - E)^2}{E}, its df k - 1, and the area to its right under a chi-square curve. This lesson adds the hypotheses, the conditions, and a conclusion about the seeds.
Reach for the chi-square goodness-of-fit test when one categorical variable is recorded on one random sample, or on the outcomes of one random process, and a claim names the proportion in every category. The question is whether the claimed distribution fits. Two categorical variables recorded on the same individuals call instead for the Tests for independence and homogeneity.
The procedure
Day 1: from claim to conclusion
Hypotheses. The null hypothesis states the whole claimed distribution, one proportion for every category. For the cross, with p standing for the long-run proportion of all offspring of this cross that have a given seed type,
and H_a: at least one of these proportions is different. The claim goes in H_0, as in Null and alternative hypotheses, because it supplies the exact numbers the expected counts are built from. The alternative has no direction and does not say that every proportion is wrong: one wrong proportion already makes H_0 false.
Conditions. The chi-square curve describes the statistic only approximately, and only when these hold.
- Random. The data come from a random sample or a random process. In a cross, each offspring’s seed type is set by chance, by which version of each gene it receives from each parent, so the counts come from a random process.
- Ten percent, when sampling without replacement. If the observations are drawn without replacement from a finite population, the sample should be at most 10\% of it, so that the draws behave as independent (the The ten percent condition). A cross draws nothing from a fixed stock, so this check does not arise there; it does for seeds taken from a company’s lot.
- Large expected counts. Every expected count is at least 5. The condition is about how well the curve approximates the statistic’s behavior when H_0 is true, and that depends on what H_0 predicts, not on what happened. An observed count of 2 or even 0 is allowed.
When a condition fails. A small expected count is handled by combining categories: merge it with another category so that the combined expected count is at least 5. The claimed proportion of the merged category is the sum of the two, its observed count is the sum of the two counts, and the degrees of freedom fall by one. Choose the merge for a reason that comes from what the categories mean (two wrinkled types become “wrinkled”), and choose it before computing anything, so that the choice cannot be steered toward a wanted result. The test then answers a coarser question, and the conclusion should say so. The other remedy is more data: the smallest expected count is n times the smallest claimed proportion. A failed Random condition has no remedy by calculation: without randomness the P-value has no meaning for any population or process.
Statistic, degrees of freedom, P-value. With k categories,
and the P-value is the area to the right of the observed \chi^2 under the chi-square curve with that df.
Decision and conclusion. Compare the P-value with the significance level \alpha, as in Deciding and concluding from a p-value. If P \le \alpha, reject H_0: there is convincing evidence that the distribution differs from the claim. If P > \alpha, fail to reject: there is not convincing evidence that it differs. Either way the conclusion names the variable, the population or process, and the claim.
Day 2: following up, and reading output
Which categories drove it. A rejection says only that the claimed distribution does not fit. The components \displaystyle \frac{(O - E)^2}{E} say where: the largest ones mark the categories farthest from the claim, relative to their size, and the sign of O - E in each says whether there were more or fewer than claimed. This follow-up is a description of the sample, not a separate test for each category.
Failing to reject. A large P-value says the counts are the kind the claim produces by chance, not that the claim is true: the same counts usually fit other nearby distributions too. Say “not convincing evidence that the distribution differs from the claim”, never “the data prove the claim” or “we accept H_0”, and do not report the components as findings.
Technology. A graphing calculator’s `χ²GOF-Test` takes the observed counts in one list, the expected counts in a second list, and the degrees of freedom. It returns \chi^2, the P-value p, the df, and `CNTRB`, the list of components in the order the categories were entered. Two checks catch most entry errors: the expected list must hold counts, adding to n, not proportions adding to 1; and the df must be k - 1, never n - 1.
Why the test is upper-tailed, and why it has no direction. Every gap is squared, so every gap raises \chi^2 whichever way it points. A statistic near 0 means a close fit, no evidence against the claim, so the P-value is always the area to the right. Squaring also removes each gap’s sign: mirror-image data give the same \chi^2 and P-value. So H_a has no direction, and direction is read from the signs of O - E in the follow-up, never from the test alone. A question about one category’s proportion in one direction belongs to the One sample test for a proportion.
Worked examples
Day 1: the pea cross, in full
Test, at \alpha = 0.05, whether the class’s 320 offspring are consistent with the 9:3:3:1 model.
SolutionHypotheses. H_0: the seed types of this cross occur in the proportions \tfrac{9}{16}, \tfrac{3}{16}, \tfrac{3}{16}, \tfrac{1}{16} (round yellow, round green, wrinkled yellow, wrinkled green). H_a: at least one of these proportions is different.
Conditions. Each offspring’s seed type comes from the random process of inheritance. Nothing is drawn without replacement from a fixed population, so the ten percent check does not arise. The expected counts are
which add to 320 and are all at least 5.
Statistic. One component per seed type:
| seed type | O | E | part |
|---|---|---|---|
| round yellow | 190 | 180 | 0.556 |
| round green | 52 | 60 | 1.067 |
| wrinkled yellow | 60 | 60 | 0 |
| wrinkled green | 18 | 20 | 0.2 |
The gaps are 10, -8, 0 and -2, so
with \text{df} = 4 - 1 = 3.
P-value. The area to the right of 1.822 under the chi-square curve with 3 degrees of freedom is about 0.610.
Conclusion. Since 0.610 > 0.05, fail to reject H_0. There is not convincing evidence that the seed types of this cross differ from the 9:3:3:1 ratio. If the model is right, about 61\% of crosses of 320 offspring would land at least this far from it by chance alone.
Day 1: a seed company’s petunia mix
Ostap’s family buys petunia seed from a company that says its garden mix is 35\% red, 25\% pink, 20\% purple, 15\% white and 5\% yellow. The company lets the class draw 400 seeds at random from a lot of 500{,}000. Every seed sprouts, and the flowers are 112 red, 94 pink, 96 purple, 68 white and 30 yellow (made-up data). Test the company’s claim at \alpha = 0.05.
SolutionHypotheses. H_0: the flower colors in this lot are 35\% red, 25\% pink, 20\% purple, 15\% white and 5\% yellow. H_a: at least one of these proportions is different.
Conditions. The seeds are a random sample from the lot. They are drawn without replacement, and 400 is far below 10\% of the lot, 50{,}000. The expected counts are
all at least 5, with sum 400.
Statistic. The gaps are -28, -6, 16, 8 and 10, so
with \text{df} = 5 - 1 = 4. The area to its right under the chi-square curve with 4 degrees of freedom is about 0.004.
Conclusion. Since 0.004 \le 0.05, reject H_0. There is convincing evidence that the color distribution of the petunias in this lot differs from the company’s stated mix. If the stated mix were right, a statistic this large would turn up in only about 4 samples of 400 in 1{,}000.
Day 1: an expected count below 5
Yaryna repeats the cross on a small plot and gets 64 offspring: 40 round yellow, 9 round green, 11 wrinkled yellow and 4 wrinkled green (made-up data). Check the conditions and test at \alpha = 0.05.
SolutionThe expected counts are 64 \cdot \tfrac{9}{16} = 36, 64 \cdot \tfrac{3}{16} = 12 twice, and 64 \cdot \tfrac{1}{16} = 4. The last is below 5, so the large expected counts condition fails. The reason is the expected 4, not the observed 4: with a claimed share of \tfrac{1}{16}, any plot of fewer than 80 offspring would fail, whatever the counts turned out to be.
Combine wrinkled green with wrinkled yellow into one category, “wrinkled”, chosen before computing anything and for a reason: both are wrinkled seeds, so the merged category still means something. Its claimed share is \tfrac{3}{16} + \tfrac{1}{16} = \tfrac{4}{16}, its expected count 16, and its observed count 11 + 4 = 15. The new hypotheses are H_0: p_{\text{RY}} = \tfrac{9}{16}, p_{\text{RG}} = \tfrac{3}{16}, p_{\text{W}} = \tfrac{4}{16}, and H_a: at least one differs. Now every expected count, 36, 12 and 16, is at least 5.
with \text{df} = 3 - 1 = 2, since there are now three categories. The P-value is about 0.533, so fail to reject H_0: there is not convincing evidence against the 9:3:4 split of round yellow, round green and wrinkled. This test can no longer compare wrinkled yellow with wrinkled green, and the conclusion must not claim that it did. Growing at least 80 offspring would have kept all four categories.
Day 2: which petunia colors drove the result
Follow up the rejection in Example 2: which colors account for the large statistic, and in which direction?
SolutionList each color’s gap and component, and the share of \chi^2 \approx 15.227 it supplies.
| color | O | E | O - E | part |
|---|---|---|---|---|
| red | 112 | 140 | -28 | 5.6 |
| pink | 94 | 100 | -6 | 0.36 |
| purple | 96 | 80 | 16 | 3.2 |
| white | 68 | 60 | 8 | 1.067 |
| yellow | 30 | 20 | 10 | 5.0 |
Red and yellow together give 5.6 + 5.0 = 10.6, about 70\% of the total, and purple adds 3.2 more. The signs give the direction: red came in below the claim (28\% of the sample against a stated 35\%), yellow above it (7.5\% against 5\%), and purple above it (24\% against 20\%). Yellow’s gap of 10 is smaller than purple’s 16 but has the larger component, because 10 extra flowers against 20 expected is half again as many, while 16 against 80 is a fifth more. In context: this lot seems to hold fewer red petunias than the company says, with yellow and purple making up the difference.
Day 2: failing to reject, said correctly
After Example 1, Solomiia writes “the test proved the 9:3:3:1 ratio”, Yaryna writes “we accept H_0”, and Ostap writes “the counts are consistent with 9:3:3:1, but they do not show that the ratio is exactly 9:3:3:1”. Test the same 320 counts against the ratio 10:3:3:1 to decide who is right.
SolutionA 10:3:3:1 ratio has 17 parts, so the expected counts are 320 \cdot \tfrac{10}{17} \approx 188.24, 320 \cdot \tfrac{3}{17} \approx 56.47 twice, and 320 \cdot \tfrac{1}{17} \approx 18.82. Then
with \text{df} = 3 and a P-value of about 0.890. The same counts sit even closer to 10:3:3:1 than to 9:3:3:1, and neither claim is rejected. Both cannot be the true ratio, so failing to reject one of them cannot be proof of it. Ostap is right; Solomiia’s “proved” and Yaryna’s “accept” both say more than the data do. (The second test is only an illustration: trying claims until one fits is not how a claim is tested.)
Day 2: reading calculator output
The three genetics classes at the school pool their crosses: 960 offspring, 571 round yellow, 163 round green, 170 wrinkled yellow and 56 wrinkled green (made-up data). Ostap puts the observed counts in one list and the expected counts in another and runs `χ²GOF-Test` with df 3. The screen shows:
| output | value |
|---|---|
| \chi^2 | 4.207 |
| p | 0.240 |
| df | 3 |
| `CNTRB` | \{1.780,\ 1.606,\ 0.556,\ 0.267\} |
Check the output against the data, state the conclusion at \alpha = 0.05, and say what would have happened had he typed the shares \tfrac{9}{16}, \tfrac{3}{16}, \tfrac{3}{16}, \tfrac{1}{16} into the expected list.
SolutionThe expected counts are \tfrac{960}{16} = 60 times 9, 3, 3 and 1: 540, 180, 180, 60, which add to 960, all at least 5. The first component is \displaystyle \frac{(571 - 540)^2}{540} = \frac{961}{540} \approx 1.780, matching the first entry of `CNTRB`. The unrounded components add to 4.207; the rounded entries add to 4.209, a gap from rounding only. The df is 4 - 1 = 3, not 959.
Since p = 0.240 > 0.05, fail to reject H_0: the pooled crosses give no convincing evidence that the seed types depart from 9:3:3:1. Round yellow and round green carry most of the statistic, but after a failure to reject those are chance-sized gaps, not a finding.
Had the expected list held the shares, adding to 1 instead of 960, the first component alone would be \displaystyle \frac{(571 - 0.5625)^2}{0.5625}, more than 500{,}000, and the P-value would print as 0.
Day 2: same statistic, opposite stories
Two made-up crosses of 320 offspring give counts A: 200, 50, 50, 20 and B: 160, 70, 70, 20 (round yellow, round green, wrinkled yellow, wrinkled green). Find \chi^2 and the P-value for each against 9:3:3:1. Then find \chi^2 for counts C: 181, 59, 60, 20, and say what each result does and does not show.
SolutionThe expected counts are 180, 60, 60, 20. Cross A has gaps 20, -10, -10, 0; cross B has -20, 10, 10, 0. Squaring gives the same four numbers, so both have components 2.222, 1.667, 1.667, 0:
with \text{df} = 3 and a P-value of about 0.135 for both. Yet A has too many round yellow seeds and B too few. The statistic and the P-value cannot tell these stories apart; only the signs of O - E can.
Cross C gives \displaystyle \chi^2 = \frac{1}{180} + \frac{1}{60} + 0 + 0 \approx 0.022, with an area to its right of about 0.999. Counts almost exactly on the claim are no evidence against it, which is why small values of \chi^2 never count toward the P-value.
Boundaries
- One categorical variable, a fully stated claim. The test compares one variable’s counts with proportions fixed before the data are seen, one for every category, adding to 1. If the proportions are instead estimated from the same data, the df is smaller than k - 1, a case this lesson does not cover.
- The conditions. A random sample or random process; at most 10\% of a finite population when sampling without replacement; every expected count at least 5. Categories must not overlap and must cover every observation.
- Large samples detect small departures. With a very large n, a gap too small to matter can still give a tiny P-value. Whether the departure matters is a separate judgment (Cautions about significance).
- Upper tail only, and no direction. The P-value is always the area to the right. A rejection says the claim does not fit; which categories are off, and which way, comes from the components and the signs of O - E, read as description.
- Two categories. With k = 2 the test has \text{df} = 1, and its \chi^2 equals the square of the one-proportion z statistic, so it is the same as the two-sided One sample test for a proportion. A one-sided question needs the z test.
Where this goes next
The Tests for independence and homogeneity uses the same statistic, conditions and upper-tail P-value for a two-way table: whether two categorical variables are associated, or whether several populations share one distribution. Its expected counts come from the table’s margins and its df is (r - 1)(c - 1); the components are read as here.
Exercises
All data are made up. Give P-values to three decimal places, using technology. Use \alpha = 0.05 unless a part says otherwise.
Day 1: carrying out the test
- Yaryna crosses pea plants for pod color and pod shape, where the model predicts 9:3:3:1 for green inflated, green constricted, yellow inflated and yellow constricted pods. For 144 offspring, state H_0 and H_a and find the expected counts.
- The 144 offspring of Exercise 1 are 88 green inflated, 22 green constricted, 25 yellow inflated and 9 yellow constricted. Check the conditions and carry out the test.
- What is the smallest number of offspring for which every expected count of a 9:3:3:1 model is at least 5?
- To test a seed mix, Ostap draws 300 seeds at random, without replacement, from a bag of 2{,}400. Which condition does this put at risk, and does it hold?
- A company says its nasturtium mix is 50\% yellow, 30\% orange and 20\% red. Of 250 seeds drawn at random from a large lot, 110 grow yellow, 90 orange and 50 red. Carry out the test at \alpha = 0.05 and at \alpha = 0.10.
- A petunia dwarf mix is said to be 40\% red, 30\% pink, 20\% purple, 6\% white and 4\% yellow. Of 100 seeds drawn at random from a large lot, the flowers are 31 red, 34 pink, 22 purple, 7 white and 6 yellow. Which condition fails? Combine categories, with a reason, and carry out the test.
- A goodness-of-fit test with 6 categories gives \chi^2 = 12.4. Find the P-value and the decision.
- Two years of the class’s dihybrid crosses together give 640 offspring: 371 round yellow, 113 round green, 118 wrinkled yellow and 38 wrinkled green. Test the 9:3:3:1 model.
Day 2: follow-up and output
- For the nasturtiums of Exercise 5, which colors contribute most to \chi^2, and in which direction?
- A pastel petunia mix is said to hold equal shares of white, lavender, pink and rose. Ostap grows 200 seeds drawn at random from a large lot and gets 46, 69, 44 and 41. His calculator shows \chi^2 = 9.88, p = 0.0196, df 3 and `CNTRB` = \{0.32,\ 7.22,\ 0.72,\ 1.62\}. State the conclusion, and say which color drove it and in which direction.
- A classmate runs the test of Exercise 10 with df 199. About what P-value appears, and why is it wrong?
- Two classes each raise 160 offspring of the dihybrid cross. Class A gets 100, 25, 25, 10 and class B gets 80, 35, 35, 10 (round yellow, round green, wrinkled yellow, wrinkled green). Find \chi^2 and the P-value for each, and say what differs between the two classes that the test does not show.
- Which sentence correctly reports Exercise 2? (a) “The test proved the 9:3:3:1 ratio.” (b) “The data show that the pod types do not follow 9:3:3:1.” (c) “There is not convincing evidence that the pod types differ from 9:3:3:1.”
- For 320 offspring, Ostap types 0.5625, 0.1875, 0.1875, 0.0625 into the expected list. What goes wrong, and what should the list hold?
- A cross of 320 offspring gives \chi^2 = 0.05 against 9:3:3:1. Find the area to the right, and say whether this is evidence against the model.
Reasoning
- Explain why the large counts condition uses expected counts, not observed counts.
- Explain why the categories to combine should be chosen for a reason before the statistic is computed.
- Explain why the P-value of a goodness-of-fit test is always an upper-tail area.
- A goodness-of-fit test rejects. Explain why the test itself cannot say which category is above or below its claimed share, and what can.
- Explain why the ten percent condition is not checked for the class’s pea crosses but is checked for the petunia lot.
- Of 100 offspring of a cross with a predicted 3:1 ratio of round to wrinkled, 68 are round and 32 wrinkled. Find \chi^2 for the goodness-of-fit test and the one-proportion z statistic for H_0\colon p = 0.75, and show that \chi^2 = z^2.
Answers
- H_0: the proportions of green inflated, green constricted, yellow inflated and yellow constricted are \tfrac{9}{16}, \tfrac{3}{16}, \tfrac{3}{16}, \tfrac{1}{16}; H_a: at least one differs. Expected: 144 \cdot \tfrac{9}{16} = 81, 27, 27, 9.
- Random process of inheritance; ten percent check does not arise; expected 81, 27, 27, 9 all at least 5. \displaystyle \chi^2 = \frac{49}{81} + \frac{25}{27} + \frac{4}{27} + 0 \approx 1.679, df 3, P-value about 0.642. Fail to reject: not convincing evidence that the pod types differ from 9:3:3:1.
- The smallest expected count is \tfrac{n}{16}, so \tfrac{n}{16} \ge 5 gives n \ge 80: at least 80 offspring.
- The ten percent condition. 10\% of 2{,}400 is 240, and 300 > 240, so it fails: the draws are too large a part of the bag to treat as independent.
- Random sample from a large lot; expected 125, 75, 50, all at least 5. \displaystyle \chi^2 = \frac{225}{125} + \frac{225}{75} + 0 = 4.8, df 2, P-value about 0.091. At \alpha = 0.05 fail to reject; at \alpha = 0.10 reject: convincing evidence at that level that the lot’s colors differ from the stated mix.
- Yellow’s expected count is 100(0.04) = 4 < 5, so large expected counts fails. Combine white and yellow as the two pale colors: share 10\%, expected 10, observed 13. With expected 40, 30, 20, 10: \displaystyle \chi^2 = \frac{81}{40} + \frac{16}{30} + \frac{4}{20} + \frac{9}{10} \approx 3.658, df 3, P-value about 0.301. Fail to reject: not convincing evidence that red, pink, purple and pale differ from 40, 30, 20, 10 percent. White and yellow are no longer compared.
- df = 5; P-value about 0.030 \le 0.05, so reject H_0.
- Expected 360, 120, 120, 40. \displaystyle \chi^2 = \frac{121}{360} + \frac{49}{120} + \frac{4}{120} + \frac{4}{40} \approx 0.878, df 3, P-value about 0.831. Fail to reject: no convincing evidence against 9:3:3:1.
- Orange gives 3.0 of the 4.8 (more than claimed, 90 against 75), yellow 1.8 (fewer, 110 against 125), red 0. As a description only, since at \alpha = 0.05 the test did not reject.
- p = 0.0196 \le 0.05: reject; convincing evidence that the colors in this lot are not in equal shares. Lavender’s component, 7.22, is most of the 9.88, and lavender came in above its share: 69 against an expected 50.
- About 1.000: the curve with 199 degrees of freedom has mean 199, so nearly all of it lies to the right of 9.88. The df counts categories, 4 - 1 = 3, not observations.
- Expected 90, 30, 30, 10. Both have \displaystyle \chi^2 = \frac{100}{90} + \frac{25}{30} + \frac{25}{30} + 0 \approx 2.778, P-value about 0.427. Class A had more round yellow than expected and class B fewer; the gaps are mirror images, which squaring hides.
- (c). (a) says a failure to reject proves the claim, and (b) states the reverse of what a large P-value says.
- The expected list adds to 1, not 320, so every component is enormous and the P-value prints as 0. The list should hold counts: 180, 60, 60, 20.
- About 0.997. No: a small statistic means the counts are very close to the model, which is no evidence against it.
- The condition asks whether the chi-square curve describes the statistic when H_0 is true, and that depends on what H_0 predicts, the expected counts. An observed count is one outcome and may be small without harm.
- A merge picked after seeing the results could be steered to move the P-value, which would then no longer mean what it says. A merge chosen for what the categories mean keeps the question honest and the merged category interpretable.
- Every gap, in either direction, increases \chi^2, which is 0 only for an exact match. So only large values are evidence against H_0, and the P-value is the chance of a value at least as large as the one observed.
- Squaring removes each gap’s sign, so mirror-image data give the same \chi^2 and P-value. The largest components show where the counts are farthest from the claim, and the sign of O - E shows the direction.
- In a cross each offspring’s type is set by chance, not drawn from a fixed stock, so there is no population to exhaust. The petunia seeds are drawn without replacement from a finite lot, so the sample must be at most 10\% of it.
- Expected 75 and 25: \displaystyle \chi^2 = \frac{49}{75} + \frac{49}{25} \approx 2.613. \displaystyle z = \frac{0.68 - 0.75}{\sqrt{0.75 \cdot 0.25 / 100}} \approx -1.617, and z^2 \approx 2.613. In general the two gaps are d and -d, and \displaystyle \frac{d^2}{np} + \frac{d^2}{n(1-p)} = \frac{d^2}{np(1-p)} = z^2.