Resistant and non resistant measures
One value in a data set is far bigger than the rest, maybe by mistake. Which summaries does it drag along with it, and which barely notice it?
Where you are 13 in Center and spread
Why this exists
Magda’s household pays a water bill every month. For ten months the bills, in dollars, are 40, 43, 36, 45, 39, 41, 34, 44, 38, 40. Now suppose a pipe under the garden had leaked in the month that cost 45 dollars, so that bill came to 115 dollars, and every other month stayed the same. Here are five summaries of the ten bills, both ways.
Two summaries did not move at all. Three moved a long way: the mean of 47 dollars is more than any ordinary month cost, and a MAD of 13.6 dollars says the bills typically sit 13.6 dollars from their center, though nine of the ten lie between 34 and 44. The 115 is an outlier by the 1.5 IQR rule: the quartiles are 38 and 43, so the fences are 30.5 and 50.5.
An extreme value may be a mistake, a rare but real case, or a member of another group. Before a summary is trusted, it helps to know which summaries one such value can push around. This lesson sorts the five.
Definition
An extreme value is a value at one end of a data set, the largest or the smallest, especially one far from the rest.
A summary is resistant when one or a few extreme values barely move it, however far out they sit. It is not resistant when a single extreme value, moved far enough, can push it as far as you like.
- Resistant: the median, the Quartiles and the five number summary Q_1 and Q_3, and the IQR, Q_3 - Q_1.
- Not resistant: the mean, the range (largest minus smallest) and the MAD (the mean of the distances from the mean).
The Results below make “barely” exact.
Intuition
The resistant summaries care about positions; the others care about sizes.
The median asks which value sits in the middle place of the sorted list. When the 45 dollar bill became 115, it was still last, so every bill kept its place and the middle places held the same two 40s. The quartiles, the middles of the two halves, stayed put for the same reason.
The mean is the total shared out evenly. The leak added 70 dollars to the total, so each of the ten shares grew by 7. Pictured as a balance point, the 115 sits far out and pulls the balance point toward it (Figure 1). The range uses the largest value directly, and the MAD measures every distance from a mean that has itself moved.
Results
For a data set of n values:
- Moving an extreme value farther out. If the largest value is made larger, or the smallest made smaller, the median does not change when n is at least 3, and Q_1, Q_3 and the IQR do not change when n is at least 6.
- The mean follows every change. Raising any one value by d raises the mean by \displaystyle \frac{d}{n}; lowering a value by d lowers the mean by \displaystyle \frac{d}{n}.
- The range follows its ends. Raising the largest value by d raises the range by d.
- The MAD has no ceiling. As one value moves farther and farther out, the MAD grows past any number.
- One new value. Adding a value larger than all the others moves the median up at most to the next value of the sorted list, however large the new value is, while it can raise the mean to any size.
Why it is true
Median and quartiles. Making the largest value larger keeps it last, so every value keeps its place. With at least 3 values no middle place is the last one, so the median is built from the same values as before. With 6 or more values each half holds at least 3 (an odd count leaves the median out, so 7 values give halves of 3), and the middle of 3 or more values is never the last, so Q_3 never uses the largest value. The smallest value and Q_1 work the same way.
Mean and range. Raising one value by d raises the total by d; shared among n values, that raises the mean by \displaystyle \frac{d}{n}, which is as large as you like when d is. The range is the largest minus the smallest, so it rises by the full d.
The MAD. The far value’s distance from the mean is one of the n distances averaged, so the MAD is at least that distance divided by n. Raising the far value by d moves the mean by only \displaystyle \frac{d}{n}, so that distance grows by \displaystyle d - \frac{d}{n}, without limit.
One new value. Put it at the end of the sorted list. If n was odd, the new middle two places hold the old median and the next value up. If n was even, the new middle place holds the upper of the two old middle values. Either way the new median lies between the old median and the next value up, and the size of the new value never enters.
Worked examples
A slip on the keyboard
Ladislav’s phone bills for nine months are, in dollars, 27, 31, 22, 36, 28, 24, 32, 23, 29. He types the 36 as 360. Find the median and the mean of the bills as typed and as they should be.
SolutionSort the correct bills: 22, 23, 24, 27, 28, 29, 31, 32, 36. Nine is odd, so the median is the fifth value, 28 dollars. The total is 252, so the mean is \displaystyle \frac{252}{9} = 28 dollars.
As typed, 360 is still last and every other bill keeps its place, so the median is still 28 dollars. The total grows by 324 to 576, so the mean becomes \displaystyle \frac{576}{9} = 64 dollars, more than any bill Ladislav really paid. A mistake in the largest value moved the mean by 36 dollars and the median not at all.
The median alone would never reveal the slip. The rule from Outliers and the 1.5 IQR rule does: as typed, Q_1 = 23.5 and Q_3 = 31.5, so the upper fence is 31.5 + 12 = 43.5 and 360 is flagged.
A new value joins
Livia’s swim club has seven swimmers, aged 9, 11, 8, 12, 10, 7, 13. Their coach, aged 46, gets into the pool with them. Find the mean and median age of the people in the pool before and after.
SolutionSorted, the ages are 7, 8, 9, 10, 11, 12, 13. The median is the fourth, 10, and the total is 70, so the mean is \displaystyle \frac{70}{7} = 10 years.
With the coach the ages are 7, 8, 9, 10, 11, 12, 13, 46. The median is halfway between the fourth and fifth, \displaystyle \frac{10 + 11}{2} = 10.5 years: it moved halfway to the next value. The total is 116, so the mean is \displaystyle \frac{116}{8} = 14.5 years, older than every swimmer.
A coach aged 76 would make the mean \displaystyle \frac{146}{8} = 18.25 years. The median would stay 10.5, as it does for any coach aged 11 or more.
A month with nobody home
Manon’s water bills for twelve months are, in dollars, 47, 6, 52, 40, 45, 54, 38, 48, 43, 50, 41, 44. The 6 dollar bill was for a month the house stood empty. Find the range and the IQR with all twelve bills and with the empty month left out.
SolutionSorted, the twelve bills split into two halves of six:
So \displaystyle Q_1 = \frac{40 + 41}{2} = 40.5 and \displaystyle Q_3 = \frac{48 + 50}{2} = 49, the IQR is 8.5 dollars, and the range is 54 - 6 = 48 dollars.
Without the empty month there are eleven bills. The median 45 is left out of both halves, which are 38, 40, 41, 43, 44 and 47, 48, 50, 52, 54, so Q_1 = 41 and Q_3 = 50, the IQR is 9 dollars, and the range is 54 - 38 = 16 dollars.
The range changed by 32 dollars, the IQR by 0.5. Leaving a value out shifts the halves by one place, so the quartiles move a little; they do not lurch.
What the MAD does
Lukasz’s phone bills for six months are, in dollars, 23, 19, 27, 21, 29, 25. In the month of the 29 dollar bill he travels abroad and the bill comes to 59. Find the MAD and the IQR both ways.
SolutionSorted: 19, 21, 23, 25, 27, 29, total 144, mean 24. The distances from 24 are 5, 3, 1, 1, 3, 5, total 18, so the MAD is \displaystyle \frac{18}{6} = 3 dollars. The halves 19, 21, 23 and 25, 27, 29 give Q_1 = 21 and Q_3 = 27, an IQR of 6 dollars.
With the 59, the total is 174 and the mean 29. Every distance is measured again from 29: 10, 8, 6, 4, 2, 30, total 60, so the MAD is \displaystyle \frac{60}{6} = 10 dollars. The upper half 25, 27, 59 still has middle 27, so the IQR is still 6 dollars.
The MAD more than tripled, for two reasons. The far bill’s own distance, 30, alone adds \displaystyle \frac{30}{6} = 5. And the mean moved up by 5, away from the five ordinary bills, whose distances grew from a total of 13 to 30.
One bill is unreadable
Magda’s eleven water bills from another year are, in dollars, 42, 37, 48, 44, 39, 51, 46, 35, 41, 47 and one bill too smudged to read. She knows it was more than 60 dollars. Which summaries can she still find?
SolutionThe smudged bill is the largest, so it goes last whatever it is: 35, 37, 39, 41, 42, 44, 46, 47, 48, 51, then the smudged one. The median is the sixth value, 44 dollars. Leaving it out, the halves are 35, 37, 39, 41, 42 and 46, 47, 48, 51 with the smudged bill, so Q_1 = 39 and Q_3 = 48, and the IQR is 9 dollars. None of these uses the smudged value.
The mean, range and MAD need its size. The ten known bills total 430, so the mean is more than \displaystyle \frac{490}{11}, about 44.5 dollars, and the range is more than 60 - 35 = 25 dollars; no more can be said. Resistance is what lets the median and IQR be found from incomplete data.
How far can one correction move each center?
Magda’s first ten bills, 34, 36, 38, 39, 40, 40, 41, 43, 44, 45 dollars, have mean and median both 40 dollars. One bill is corrected to a new amount. (a) What would the 45 have to be corrected to for the mean to become 45? (b) Can correcting any one bill make the median 45?
Solution(a) A mean of 45 over ten bills needs a total of 450. The total is 400, so it must grow by 50: the 45 becomes 95 dollars.
(b) No. Six bills are 40 dollars or less. Correcting one bill leaves at least five of them, so the fifth value of the new sorted list is at most 40. Seven bills are 41 dollars or less, so at least six remain and the sixth value is at most 41. The median is at most \displaystyle \frac{40 + 41}{2} = 40.5 dollars, reached by raising one of the six lowest bills above 41. One correction can raise the mean as far as you like, but moves the median half a dollar at most.
Boundaries
Very small data sets. With 2 values the median is the mean of both, so it moves with the larger. With fewer than 6 values a half has at most 2 values, so Q_3 uses the largest value and the IQR can move (Exercise 13).
Resistant is not frozen. A changed middle value can move the median, and so can many changed values or a whole group of large ones. Resistance is about one or a few extreme values.
Resistant does not reveal mistakes. The median ignored the slip in Example 1, so it could not warn of it. Look at the data, or use the 1.5 IQR rule, as well.
Resistant is not always better. A total is always the mean times the count: a year’s water costs twelve times the monthly mean bill, not twelve times the median.
Where this goes next
The pairs travel together: the median with the IQR, and the mean with the MAD. Comparing two distributions uses that pairing, and Mean, median and skew: choosing the center shows the long tail of a skewed data set pulling the mean toward it. Later summaries are sorted the same way: the standard deviation is not resistant, and neither is the correlation coefficient (Properties of the correlation coefficient).
Exercises
Before and after
- Lukasz’s phone bills for five months are 18, 21, 20, 24, 17 dollars. The 24 is typed as 74. Find the median and the mean both ways.
- Manon’s water bills for six months are 31, 35, 28, 33, 30, 37 dollars. Find the range and the IQR. Then find them again if the 37 had been 97.
- Livia’s phone bills for five months are 14, 10, 18, 12, 16 dollars. Find the mean and the MAD. Then the 18 becomes 43; find them again.
- Magda’s water bills for eight months are 52, 44, 58, 50, 62, 48, 56, 54 dollars. Find the median, IQR, mean, range and MAD. Then find all five again if the 62 had been 142.
- Ladislav’s phone bills for seven months are 47, 54, 39, 50, 45, 52, 42 dollars. A refund turns the 39 into 4. Find the median, mean and range both ways.
- Five swimmers are aged 12, 15, 13, 14, 13. A coach aged 41 joins them. Find the median and mean age before and after.
From a table
Table A shows the ages of the 15 swimmers in a class.
| Age | 8 | 9 | 10 | 11 | 12 |
|---|---|---|---|---|---|
| Swimmers | 2 | 4 | 5 | 3 | 1 |
- From Table A, find the median and mean age. A coach aged 45 joins the class; find both again.
Which summaries move
- A data set has 15 values. Its largest value is raised by 30 and stays the largest. By how much do the mean, median, range and IQR change?
- Twelve phone bills have a mean of 30 dollars. One bill of 25 dollars was entered as 85. What mean was reported?
- Manon’s nine water bills are 43, 50, 34, 52, 45, 38, 46, 41 dollars and one unreadable bill known to be more than 60 dollars. Find the median, Q_1, Q_3 and the IQR. What can be said about the range?
- Livia’s water bills for eight months are 21, 14, 27, 17, 24, 19, 16, 22 dollars. What ninth bill would make the mean of the nine 25 dollars? What would their median be?
- Lukasz’s ten phone bills are 30, 24, 46, 28, 35, 21, 38, 30, 25, 33 dollars. (a) What would the 46 have to be corrected to for the mean to become 35? (b) If any one bill is corrected, what is the largest the median could become?
- Find the IQR of 2, 5, 6, 9. Then change the 9 to 19 and find it again. Why does this not contradict the Results?
Explain your reasoning
- Give a list of five values and a change to one value that moves the median. What kind of value did you change?
- Explain why the median of a list of at least 3 values does not change when the largest value is made larger.
- Explain why the range can never be resistant.
- Using the mean as a balance point, explain why moving one value far out moves the mean.
- True or false: “If the mean of a data set is far from its median, some value must be a mistake.” Explain.
- Can one added value make the mean as large as you like? The median? Explain.
Answers
- Correct: median 20, mean 20 dollars. As typed: median 20, mean 30 dollars. Sorted: 17, 18, 20, 21, 24, total 100; the typed total is 150, and 74 is still last.
- Range 9, IQR 5 dollars; then range 69, IQR 5 dollars. Sorted 28, 30, 31, 33, 35, 37: Q_1 = 30 and Q_3 = 35, and 35 is still the middle of 33, 35, 97.
- Mean 14, MAD 2.4 dollars; then mean 19, MAD 9.6 dollars. The distances from 14 total 12; the new total is 95, and the distances from 19 are 9, 7, 5, 3, 24, total 48.
- Before: median 53, IQR 8, mean 53, range 18, MAD 4.5. After: median 53, IQR 8, mean 63, range 98, MAD 19.75. Sorted 44, 48, 50, 52, 54, 56, 58, 62: Q_1 = 49, Q_3 = 57, total 424, distances from 53 total 36. After, the total is 504 and the distances from 63 total 158.
- Before: median 47, mean 47, range 15 dollars. After: median 47, mean 42, range 50 dollars. The total falls by 35 to 294, so the mean falls by \displaystyle \frac{35}{7} = 5.
- Before: median 13, mean 13.4 years. After: median 13.5, mean 18 years. Sorted 12, 13, 13, 14, 15, 41; the total is 67 + 41 = 108.
- Before: median 10, mean 9.8 years. After: median 10, mean 12 years. The running totals are 2, 6, 11, 14, 15, so the eighth and ninth ages are 10. The total is 147, then 192 over 16.
- The mean rises by 2, the range by 30, and the median and IQR do not change. The mean rises by \displaystyle \frac{30}{15}; with 15 values the median and quartiles never use the largest.
- 35 dollars. The total was 360 and grew by 60, and \displaystyle \frac{420}{12} = 35.
- Median 45, Q_1 = 39.5, Q_3 = 51, IQR 11.5 dollars; the range is more than 26 dollars. The unreadable bill goes last, so the halves are 34, 38, 41, 43 and 46, 50, 52 with it. The range exceeds 60 - 34.
- 65 dollars, with median 21 dollars. The eight total 160, and nine bills with mean 25 total 225. Sorted, 14, 16, 17, 19, 21, 22, 24, 27, 65 has fifth value 21.
- (a) 86 dollars: the total 310 must become 350. (b) 31.5 dollars. Six bills are 30 or less and seven are 33 or less, so after one correction the fifth value is at most 30 and the sixth at most 33; raising the 21 above 33 gives \displaystyle \frac{30 + 33}{2}.
- IQR 4, then 9. The halves are 2, 5 and 6, 9, so Q_3 = 7.5, then 12.5. With four values each half has two, so Q_3 uses the largest; the Results need at least 6 values.
- For example 3, 5, 7, 8, 9: changing the 3 to 10 gives 5, 7, 8, 9, 10, median 8, not 7. The changed value crossed from below the middle to above it, which an extreme value moved farther out on its own side never does.
- The largest value stays last, so every value keeps its place. With at least three values no middle place is the last, so the median is made of the same values as before.
- The range is the largest value minus the smallest, so moving the largest up by any amount moves the range up by the same amount, as far as you like.
- The mean is where the distances on the low side total the same as those on the high side. Moving one value far out adds to the high side’s total, so the balance point must move toward it until the totals match again.
- False. A rare but real value, such as a month with a leak, or a long tail of real values can make the gap. Only checking the record shows whether a value is a mistake.
- The mean, yes: the new value adds its whole size to the total. The median, no: it moves at most to the next value of the sorted list above the old median, however large the new value is.