Statistics

Expected counts and the chi-square statistic

A beekeeper expects bees to visit four flower colors equally often, and the counts come back uneven. How can one number say how far a whole set of counts sits from what a claim predicts, and how large that number tends to be when the claim is true?

You need

The t distribution and degrees of freedomStatistics Null and alternative hypothesesStatistics Joint and marginal frequenciesMath 8

Unlocks

Goodness of fit testStatistics

Where you are 3 in Chi-square tests

Statistics7

Statistical inference8

Chi-square tests3

Expected counts and the chi-square statistic

Goodness of fit test Tests for independence and homogeneity

When you reach for it

Nikos keeps bees on Crete. In a test garden he planted equal areas of yellow, white, purple and blue flowers, and he recorded which color each of 200 honeybee visits went to (made-up data):

color	visits
yellow	62
white	41
purple	48
blue	49
total	200

If the bees have no preference, each color should get about a quarter of the visits, 50 each. The counts are not 50, but chance alone never gives exactly 50 either. Before anyone can ask whether these gaps are too large to be chance, the four gaps have to be gathered into one number. A Test statistic for one proportion did that for a single gap; here there are four gaps at once, and the same need arises for a whole two-way table of counts.

Reach for this lesson whenever counts in categories are set against a claim: a claimed set of proportions for one categorical variable, or a claim that two categorical variables are not associated. Day 1 finds what the claim predicts, the expected counts, and combines the gaps into the chi-square statistic. Day 2 describes the chi-square distributions, which say how large that statistic tends to be when the claim is true.

The procedure

Day 1: expected counts and the statistic

The claim is a null hypothesis. In Null and alternative hypotheses the surf school’s claim, p = 0.80, was a null hypothesis about one proportion. Nikos’s claim is a null hypothesis about four proportions at once:

\begin{aligned} H_0 \colon &p_{\text{yellow}} = p_{\text{white}} \\ &= p_{\text{purple}} = p_{\text{blue}} = 0.25 \end{aligned}

Everything below is worked out as if H_0 were true.

Expected counts for one variable. If a category has claimed proportion p and there are n observations, the claim predicts

E = n \cdot p

in that category, because a share p of n is np. For Nikos, E = 200 \times 0.25 = 50 for each color. An expected count is a long-run average, not a count anyone must see, so it need not be a whole number: 150 \times 0.07 = 10.5 is a perfectly good expected count. The expected counts always add to n, because the claimed proportions add to 1.

Expected counts for a two-way table. Here the claim is that the row variable and the column variable are not associated. In Association in a two way table and segmented bar graphs, no association meant every row has the same conditional shares, and that lesson already found the counts such a table would have. The rule behind them: if there is no association, every row should split across the columns in the same proportions as the whole table does. A column holds the share

\frac{\text{column total}}{\text{grand total}}

of the whole table, so a row with a given total is expected to put that same share of its individuals in that column:

\begin{aligned} E &= \text{row total} \cdot \frac{\text{column total}}{\text{grand total}} \\ &= \frac{(\text{row total})(\text{column total})}{\text{grand total}} \end{aligned}

The rule uses only the marginal totals, so the expected table has the same row totals and column totals as the observed one.

The statistic. In each category or cell, take the gap O - E, square it, divide by E, and add over all of them:

\chi^2 = \sum \frac{(O - E)^2}{E}

Each step has a reason.

Why square. Observed and expected counts have the same total, so the gaps O - E always add to 0: a plain sum of gaps would be 0 for every data set. Squaring makes every gap count as a positive amount, whichever way it points.
Why divide by E. A gap of 5 visits is large when 15 were expected and small when 500 were. Dividing by E measures each squared gap against the size of its expected count, so categories of different sizes are put on one scale.

Components. Each term \dfrac{(O - E)^2}{E} is a component of the statistic. The largest components show where the data and the claim disagree most, and the sign of O - E in those cells says which way: more than expected, or fewer. The total \chi^2 says only how far the counts are from the claim, never in which direction, so a full description reads the components too.

Day 2: the chi-square distributions

The family. For each positive whole number of degrees of freedom there is one chi-square distribution. Every one of them

takes only values 0 and above, as the statistic does, since it is a sum of squares divided by positive counts,
is skewed to the right (see Symmetric and skewed distributions),
has mean equal to its degrees of freedom and standard deviation \displaystyle \sqrt{2\,\text{df}},
has its peak at \text{df} - 2 when \text{df} \ge 2, and becomes more nearly symmetric as df grows.

Figure 1Chi-square curves for 3, 5 and 10 degrees of freedom. Each starts at 0, has a long right tail, and is centered further right and more nearly symmetric as the degrees of freedom grow.

Like the The t distribution and degrees of freedom, this is a family indexed by degrees of freedom, but its curves are not centered at 0 and they are never symmetric.

What the curve describes. When the claimed model is true and the expected counts are large enough, the values of the chi-square statistic over all possible samples follow, approximately, the chi-square distribution with the right degrees of freedom. “Large enough” is one of the conditions of the next two lessons.

Degrees of freedom. The df counts how many counts are free to vary once the totals the expected counts were built from are fixed.

One variable with k categories: \text{df} = k - 1. The counts must add to n, so once k - 1 of them are known the last is forced.
A table with r rows and c columns: \text{df} = (r - 1)(c - 1). Every row must add to its row total and every column to its column total. Fill in the first r - 1 rows and first c - 1 columns freely; the last cell of each of those rows is then forced by its row total, and every cell of the last row is forced by its column total. That leaves (r - 1)(c - 1) free cells.

The degrees of freedom count categories or cells, never the number of observations.

Upper-tail areas. A large statistic means the counts sit far from the claim, so the area of interest is the area to the right of the statistic. A chi-square table gives values that cut off a few chosen upper-tail areas:

\text{df}	0.25	0.20	0.10
1	1.323	1.642	2.706
2	2.773	3.219	4.605
3	4.108	4.642	6.251
4	5.385	5.989	7.779
5	6.626	7.289	9.236
6	7.841	8.558	10.645

\text{df}	0.05	0.025
1	3.841	5.024
2	5.991	7.378
3	7.815	9.348
4	9.488	11.143
5	11.070	12.833
6	12.592	14.449

A statistic that falls between two entries of its row has an upper-tail area between the two column heads. Technology gives the area itself: on a graphing calculator `chi2cdf(lower, upper, df)` returns the area between two values (a very large number such as 10^{99} stands in for “no upper end”), and in a spreadsheet `CHISQ.DIST.RT(x, df)` returns the area to the right of x. This area is the P-value of the tests that follow.

Worked examples

Example 1

Day 1: the bee garden

Find the expected counts, the components and the chi-square statistic for Nikos’s 200 visits, and say which colors contribute most.

Solution

Under H_0 each color has proportion 0.25, so each expected count is 200 \times 0.25 = 50. The gaps O - E are 12, -9, -2 and -1; they add to 0, as they must.

color	O	(O - E)^2	part
yellow	62	144	2.88
white	41	81	1.62
purple	48	4	0.08
blue	49	1	0.02

Since every E is 50, the components can be added over one denominator:

\chi^2 = \frac{144 + 81 + 4 + 1}{50} = \frac{230}{50} = 4.6

Yellow gives 2.88 of the 4.6, about 63\% of it, and yellow and white together give 4.5. Purple and blue are almost exactly on the claim. So whatever distance there is between the counts and “no preference” comes from more visits to yellow than expected and fewer to white.

Example 2

Day 1: unequal claimed proportions

Yiannis plants a second garden with 40\% of its area yellow, 30\% white, 20\% purple and 10\% blue, and claims that visits follow the areas. Of 150 visits, 54 go to yellow, 50 to white, 26 to purple and 20 to blue (made-up data). Find the expected counts and the statistic, and say which color contributes most.

Solution

The expected counts are n \cdot p for each claimed share:

\begin{gathered} 150(0.40) = 60, \quad 150(0.30) = 45, \\ 150(0.20) = 30, \quad 150(0.10) = 15 \end{gathered}

They add to 150. The gaps are -6, 5, -4 and 5, and the components are

\begin{gathered} \frac{36}{60} = 0.6, \quad \frac{25}{45} \approx 0.556, \\ \frac{16}{30} \approx 0.533, \quad \frac{25}{15} \approx 1.667 \end{gathered}

so \displaystyle \chi^2 = \frac{151}{45} \approx 3.356. Blue contributes most, about half of the total, although yellow’s gap of 6 is the largest. Blue’s 5 extra visits are a third of the 15 it was expected to get; yellow’s 6 missing visits are a tenth of its 60. Dividing by E is what makes the small category count for more here.

Example 3

Day 1: expected counts in a two-way table

Despina keeps 120 hives, 72 on a hillside and 48 in a valley. She opens each hive once in May and records the colony’s behavior (made-up data):

	hillside	valley	total
calm	42	18	60
restless	18	18	36
defensive	12	12	24
total	72	48	120

Find the expected counts if location and behavior are not associated, and explain the first one.

Solution

Overall, \displaystyle \frac{60}{120} = 50\% of colonies were calm. With no association the hillside should be calm at the same rate, so 50\% of its 72 hives, 36, are expected to be calm. That is the rule \displaystyle \frac{72 \cdot 60}{120} = 36. The other cells follow the same way:

\begin{aligned} \text{hillside: } &\tfrac{72 \cdot 36}{120} = 21.6, \quad \tfrac{72 \cdot 24}{120} = 14.4 \\ \text{valley: } &\tfrac{48 \cdot 60}{120} = 24, \quad \tfrac{48 \cdot 36}{120} = 14.4, \\ &\tfrac{48 \cdot 24}{120} = 9.6 \end{aligned}

expected	hillside	valley	total
calm	36	24	60
restless	21.6	14.4	36
defensive	14.4	9.6	24
total	72	48	120

The expected table keeps the observed row and column totals. Its two columns also split the same way, as no association requires: the hillside and the valley are each 50\% calm, 30\% restless and 20\% defensive. The values 21.6 and 14.4 are not whole numbers, which is fine for an average.

Example 4

Day 1: the statistic for the table, and where it comes from

Find the chi-square statistic for Despina’s table and describe which cells contribute most.

Solution

One component per cell, using the observed and expected tables:

\begin{aligned} \text{hillside calm: } &\frac{(42 - 36)^2}{36} = 1.0 \\ \text{hillside restless: } &\frac{(18 - 21.6)^2}{21.6} = 0.6 \\ \text{hillside defensive: } &\frac{(12 - 14.4)^2}{14.4} = 0.4 \\ \text{valley calm: } &\frac{(18 - 24)^2}{24} = 1.5 \\ \text{valley restless: } &\frac{(18 - 14.4)^2}{14.4} = 0.9 \\ \text{valley defensive: } &\frac{(12 - 9.6)^2}{9.6} = 0.6 \end{aligned}

The six components add to \chi^2 = 5.0. The two largest are both in the calm column: the valley had 6 fewer calm colonies than expected (component 1.5) and the hillside 6 more (component 1.0). The gaps are the same size, but the valley’s expected count, 24, is smaller than the hillside’s 36, so its component is larger. The pattern the components point to is that hillside colonies were calmer than no association predicts.

Example 5

Day 2: counting degrees of freedom

Find the degrees of freedom for Nikos’s garden and for Despina’s table, and show which counts are forced.

Solution

Nikos has k = 4 colors, so \text{df} = 4 - 1 = 3. Given 62 yellow, 41 white and 48 purple out of 200, blue must be 200 - 151 = 49: only three counts were free.

Despina’s table has 3 rows and 2 columns, so \text{df} = (3 - 1)(2 - 1) = 2. Keep her totals and choose only the hillside calm count, 42, and the hillside restless count, 18. Then the rest are forced:

\begin{aligned} \text{hillside defensive} &= 72 - 42 - 18 = 12 \\ \text{valley calm} &= 60 - 42 = 18 \\ \text{valley restless} &= 36 - 18 = 18 \\ \text{valley defensive} &= 24 - 12 = 12 \end{aligned}

Two free cells out of six, which is (3 - 1)(2 - 1).

Example 6

Day 2: an upper-tail area, by table and by technology

Find the area to the right of Nikos’s statistic, \chi^2 = 4.6, under the chi-square curve with \text{df} = 3, first bracketed from the table and then exactly. Do the same for Despina’s \chi^2 = 5.0 with \text{df} = 2.

Solution

In the row for \text{df} = 3, 4.108 has upper area 0.25 and 4.642 has 0.20. Since 4.6 lies between them, just below 4.642, the area to its right is between 0.20 and 0.25, close to 0.20. Technology gives it: `chi2cdf(4.6, 10^99, 3)` or `CHISQ.DIST.RT(4.6, 3)` returns about 0.204.

Figure 2The upper-tail area beyond the beekeeper’s statistic, 4.6, under the chi-square curve with 3 degrees of freedom is about 0.204.

The mean of this distribution is 3, and 4.6 is above it but well inside the bulk of the curve: about one sample in five would give a statistic at least this large if the bees had no preference.

For Despina, the \text{df} = 2 row has 4.605 at 0.10 and 5.991 at 0.05, so the area to the right of 5.0 is between 0.05 and 0.10. Technology gives about 0.082.

Example 7

Day 2: a simulation of the statistic

Nikos’s laptop builds 10{,}000 imaginary gardens in which each of 200 visits goes to one of the four colors with probability 0.25 each, so H_0 is true by construction, and computes \chi^2 for each (made-up simulation). The mean of the 10{,}000 statistics is 3.019; 2{,}118 of them are 4.6 or more, and 506 are 7.815 or more. Compare these with the chi-square distribution with \text{df} = 3.

Solution

The distribution’s mean is its df, 3, and the simulated mean is 3.019. The share at or above 4.6 is \displaystyle \frac{2118}{10000} \approx 0.212, close to the curve’s 0.204. The share at or above 7.815 is 0.0506, against the curve’s 0.05 from the table.

Figure 3Ten thousand simulated gardens of 200 visits with every color equally likely (made-up simulation). The histogram of their chi-square statistics follows the chi-square curve with 3 degrees of freedom. Values above 16 are too rare to show.

The histogram shows the same agreement: tallest between 1 and 2, a long right tail, and bars that follow the curve. The small differences have two sources. A rerun would give slightly different counts, and the statistic itself only takes certain values (the counts are whole numbers), while the curve is smooth. A lump of probability sits exactly at 4.6, and “at or above” takes all of it. So the chi-square distribution is an approximation, a good one when the expected counts are as large as Nikos’s 50. The simulation also gives the plain meaning of the 0.204: with no preference at all, a statistic of 4.6 or more is an ordinary outcome.

Boundaries

Counts only. The statistic is computed from counts, never from percentages, proportions or means. The same percentages from twice as many observations give twice the statistic, because every gap doubles, its square grows fourfold, and each E only doubles.
Expected counts large enough. The chi-square curve fits the statistic only when every expected count is reasonably large. A small expected count makes its component swing widely from small gaps. The next two lessons state the condition.
Categories that cover every observation once. Each observation goes into exactly one category or cell, and the categories must cover all of them, so that observed and expected totals agree. The observations also need to be independent of each other, a condition the tests check.
Distance, not direction. \chi^2 is 0 only when every count equals its expectation, and it grows with any gap in any direction. The direction of the disagreement is read from the signs of O - E in the largest components, never from the total.
Upper tail only. Only large values of the statistic say the counts are far from the claim, so only the area to the right is used.
The right df. Degrees of freedom come from the number of categories or cells, never from n. The curve with the wrong df gives the wrong area.

Where this goes next

The Goodness of fit test uses the one-variable statistic and the curve with k - 1 degrees of freedom to judge a claimed set of proportions. The Tests for independence and homogeneity does the same for a two-way table, with (r - 1)(c - 1) degrees of freedom. Both lessons add the hypotheses, the conditions and the conclusion; in short, a large statistic with a small upper-tail area would suggest that the claimed model does not fit the counts, and the components then say where.

Exercises

All data are made up. Give areas to three decimal places, using technology unless a part says otherwise.

Day 1: expected counts and the statistic

An herb patch has area shares of 50\% thyme, 30\% sage and 20\% oregano. For 240 visits, find the expected counts if visits follow the areas.
In the herb patch of Exercise 1 the 240 visits were 130 thyme, 62 sage and 48 oregano. Find the chi-square statistic and say which herb contributes most.
On a cool morning the four-color garden gets 120 visits: 36 yellow, 30 white, 24 purple and 30 blue. Find the chi-square statistic for equal shares.
In a two-way table with grand total 150, a cell’s row total is 90 and its column total is 40. Find its expected count under no association.
Of Despina’s 80 hives at a second site, 50 are wooden and 30 polystyrene. Of the wooden hives, 16 swarmed this spring and 34 did not; of the polystyrene hives, 4 swarmed and 26 did not. Find the four expected counts under no association between hive type and swarming.
For the hives of Exercise 5, find the chi-square statistic, and say which cell contributes most and in which direction.
Five herbs grow in equal areas. Of 100 visits, 25, 18, 20, 22 and 15 go to the five herbs. Find the chi-square statistic for equal shares.

Day 2: the chi-square distributions

Give the degrees of freedom for (a) one variable with 6 categories, (b) a table with 3 rows and 4 columns, (c) a table with 2 rows and 2 columns.
Find the area to the right of \chi^2 = 7.2 with \text{df} = 4.
Find the upper-tail area for the statistic of Exercise 2.
Find the upper-tail area for the statistic of Exercise 6.
Using only the table, bracket the area to the right of \chi^2 = 10.1 with \text{df} = 4. Then find it exactly.
Find the value with upper-tail area 0.05 for \text{df} = 3 and for \text{df} = 2.
Give the mean and standard deviation of the chi-square distribution with \text{df} = 6.
Yiannis runs a simulation of 1{,}000 gardens of 200 visits each, with four equally likely colors, and 52 of the 1{,}000 statistics are 7.8 or above. Compare this share with the area to the right of 7.8 under the chi-square curve with \text{df} = 3.
A table has row totals 30, 40, 50 and column totals 35, 45, 40. The first two rows begin 10, 12 and 15, 14. Fill in the other five cells, and say what this shows about the degrees of freedom.

Reasoning

Explain why the gaps O - E always add to 0 for one categorical variable, and what this means for a statistic built from them.
Explain why each squared gap is divided by E rather than left as it is.
Can a chi-square statistic be negative? What does \chi^2 = 0 say about the data?
Nikos repeats his count with 400 visits and gets 124 yellow, 82 white, 96 purple and 98 blue. Find \chi^2 and its upper-tail area, and compare with the 200 visits.
Explain, with a 2 by 2 table, why its degrees of freedom are 1.
Explain why the expected count in a two-way table is the row total times the column total, divided by the grand total, when there is no association.

Answers

240(0.50) = 120 thyme, 240(0.30) = 72 sage, 240(0.20) = 48 oregano.
The components are \displaystyle \frac{100}{120} \approx 0.833, \displaystyle \frac{100}{72} \approx 1.389 and \displaystyle \frac{0}{48} = 0, so \chi^2 \approx 2.222. Sage contributes most: its gap of 10 is the same size as thyme’s, but against a smaller expected count. It got fewer visits than expected.
Each E = 30, gaps 6, 0, -6, 0, so \displaystyle \chi^2 = \frac{36 + 0 + 36 + 0}{30} = 2.4, split equally between yellow and purple.
\displaystyle E = \frac{90 \cdot 40}{150} = 24.
Swarmed: \displaystyle \frac{50 \cdot 20}{80} = 12.5 wooden and \displaystyle \frac{30 \cdot 20}{80} = 7.5 polystyrene. Did not swarm: \displaystyle \frac{50 \cdot 60}{80} = 37.5 wooden and \displaystyle \frac{30 \cdot 60}{80} = 22.5 polystyrene.
Every gap is 3.5 in size, so each component is \displaystyle \frac{12.25}{E}: 0.980, 0.327, 1.633 and 0.544, and \chi^2 \approx 3.484. The largest component, 1.633, is polystyrene hives that swarmed: 4 against 7.5 expected, fewer than no association predicts.
Each E = 20: \displaystyle \chi^2 = \frac{25 + 4 + 0 + 4 + 25}{20} = 2.9.
(a) 6 - 1 = 5. (b) (3 - 1)(4 - 1) = 6. (c) (2 - 1)(2 - 1) = 1.
`CHISQ.DIST.RT(7.2, 4)`: about 0.126.
\chi^2 \approx 2.222 with \text{df} = 3 - 1 = 2: about 0.329.
\chi^2 \approx 3.484 with \text{df} = (2 - 1)(2 - 1) = 1: about 0.062.
In the \text{df} = 4 row, 9.488 has upper area 0.05 and 11.143 has 0.025, so the area is between 0.025 and 0.05. Exactly, about 0.039.
7.815 for \text{df} = 3 and 5.991 for \text{df} = 2.
Mean 6; standard deviation \displaystyle \sqrt{12} \approx 3.464.
The simulated share is \displaystyle \frac{52}{1000} = 0.052; the curve’s area is about 0.050. They agree up to chance variation in 1{,}000 runs, where a share near 0.05 typically wanders by about \displaystyle \sqrt{0.05(0.95)/1000} \approx 0.007.
Row 1 ends 30 - 22 = 8; row 2 ends 40 - 29 = 11; row 3 is 35 - 25 = 10, 45 - 26 = 19 and 50 - 29 = 21 (and column 3 checks: 8 + 11 + 21 = 40). Four cells, (3 - 1)(3 - 1), were free and fixed all the others, so \text{df} = 4.
The observed counts add to n and the expected counts add to n (the claimed proportions add to 1), so the gaps add to n - n = 0. A statistic that simply added the gaps would be 0 for every data set, so the gaps must be made positive, which squaring does.
The same gap means more in a small category than in a large one: 5 extra against 15 expected is a third more, against 500 only 1\%. Dividing by E measures each squared gap relative to its expected count, so a cell’s component reflects how surprising its gap is, not just how big the category is.
No: each component is a square divided by a positive count, so each is 0 or more. \chi^2 = 0 only when every observed count equals its expected count exactly.
E = 100 each, and the squared gaps add to 920, so \displaystyle \chi^2 = \frac{920}{100} = 9.2, double the 4.6, with area about 0.027 instead of 0.204. The shares are identical, but the same shares from more visits are much harder to explain by chance.
With both row totals and both column totals fixed, choosing the top-left cell fixes the top-right (from the first row total), the bottom-left (from the first column total) and then the bottom-right (from either remaining total). One free cell: (2 - 1)(2 - 1) = 1.
With no association, every row splits across the columns in the same shares as the whole table, and a column’s share of the whole table is its column total divided by the grand total. A row with a given total is then expected to put that share of its members in the column, which is the row total times the column total, divided by the grand total.