Standardizing a data set
Two data sets sit on different scales, in different units or with different spreads. How can every value in each be put on one common scale, where the middle is always 0 and a step of 1 always means the same thing?
Where you are 13 in Center and spread
When you reach for it
Raw numbers carry their scale with them. A disc golf hole 70 meters long, dough that takes 36 minutes to rise and a score of 9 out of 10 cannot be set side by side: the units differ, and even within one unit a “big” value depends on what is typical for that set. Two judges can both mark out of 10 and still mean different things by a 9 if one of them spreads marks widely and the other keeps them close together.
Standardizing a data set rewrites every value as a distance from the set’s mean, measured in standard deviations. Whatever the data were, the new set has mean 0 and standard deviation 1, and it has no unit at all. Reach for it when you want to:
- put two or more data sets on one common scale, so their values can be compared or combined fairly;
- read every value at a glance as “above or below typical, and by how much”;
- prepare data for a method that expects that common scale, such as finding a correlation from standardized values or reading a normal curve as the standard normal curve.
Describing one value’s position by its standardized value is the subject of z-scores and the empirical rule. This lesson does the whole set at once and asks what the new set looks like. All the data here are made up.
The procedure
Standardizing uses the two moves of shifting and rescaling a data set, one after the other, chosen so that the result has mean 0 and standard deviation 1.
For each value x in a population with mean \mu and standard deviation \sigma, the standardized value is
For a sample, the mean is \bar{x} and the standard deviation is s, the version that divides by n - 1, as in Computing a standard deviation, sample and population:
The two formulas do the same thing with whichever mean and standard deviation suit the data. Which one to use is decided the same way as there: if the values are the whole group being described, use \mu and \sigma; if they stand in for a bigger group, use \bar{x} and s. The rest of this section is written for a population; every step works word for word with \bar{x} and s.
Step one: subtract the mean
Subtracting \mu from every value is a shift by the constant -\mu. A shift moves the mean by the same amount and leaves the standard deviation alone, so the shifted values x - \mu, the deviations, have
The center is now at 0. Each deviation says how far its value sits from the mean, in the data’s own unit, with a sign: negative below the mean, positive above.
Step two: divide by the standard deviation
Dividing every deviation by \sigma multiplies it by \displaystyle \frac{1}{\sigma}, a positive constant. A positive rescale multiplies the mean and the standard deviation by that constant, so the new values z have
That is the whole reason for the two numbers chosen. Subtracting the mean, and nothing else, puts the center at 0. Dividing by the standard deviation, and nothing else, makes the spread 1. The order matters: dividing first and then subtracting \mu would not center the data, because after the division the mean is \displaystyle \frac{\mu}{\sigma}, not \mu.
What the new set looks like
A standardized value counts standard deviations. z = 1.5 means one and a half standard deviations above the mean; z = -0.5 means half a standard deviation below it; z = 0 is the mean itself. Reading the formula backwards gives the original value:
It has no unit. x - \mu is in the data’s unit, and so is \sigma, so the unit cancels in the division. Meters over meters is a plain number. That is what lets values from different data sets stand on one scale.
The order and the shape are kept. Both moves are a shift and a rescale by a positive number, so the smallest value stays smallest, the median stays in the middle, and the dot plot is the same picture with new numbers on its axis. A set that is skewed before is skewed in the same direction after; a gap or a cluster stays a gap or a cluster. Standardizing never makes data symmetric, and never makes them normal (Figure 2, in Example 2).
Every position measure goes through the same formula. The median, the quartiles, the minimum and the maximum each become (value - \mu) \div\, \sigma, just as each data value does. Every measure of spread, the range and the interquartile range included, is divided by \sigma, since the \mu cancels whenever one value is taken from another.
Two free checks. Because the deviations add to 0, the standardized values add to 0. And the squares of the standardized values add to a fixed number: for a population of N values,
since \sum (x - \mu)^2 = N\sigma^2 is what the variance means. For a sample of n values, \sum (x - \bar{x})^2 = (n - 1)s^2, so \sum z^2 = n - 1. A list of standardized values that fails either check has an arithmetic slip somewhere.
Going on to any mean and standard deviation. Once the set has mean 0 and standard deviation 1, the rule y = m + dz, with d > 0, gives a set with mean m and standard deviation d, by the same two moves again. Reports that want every set on a familiar scale, such as mean 50 and standard deviation 10, do exactly this.
Worked examples
Standardizing a whole course
A small disc golf course has five holes, of lengths 52, 58, 61, 64 and 70 meters. Standardize the lengths, and confirm that the new set has mean 0 and standard deviation 1.
SolutionThe five holes are the whole course, the group being described, so they are a population: use \mu and \sigma, dividing by N = 5.
The mean is \displaystyle \mu = \frac{52 + 58 + 61 + 64 + 70}{5} = \frac{305}{5} = 61 meters. The deviations are -9, -3, 0, 3, 9, which add to 0 as deviations must. Their squares are 81, 9, 0, 9, 81, adding to 180, so
Divide each deviation by 6:
| length x (m) | x - \mu (m) | \displaystyle z = \frac{x - \mu}{\sigma} |
|---|---|---|
| 52 | -9 | -1.5 |
| 58 | -3 | -0.5 |
| 61 | 0 | 0 |
| 64 | 3 | 0.5 |
| 70 | 9 | 1.5 |
The standardized set is -1.5, -0.5, 0, 0.5, 1.5, with no unit. The longest hole is one and a half standard deviations longer than the mean hole; the middle hole is exactly average.
Check directly. The mean of the new set is \displaystyle \frac{-1.5 - 0.5 + 0 + 0.5 + 1.5}{5} = 0. Its squares are 2.25, 0.25, 0, 0.25, 2.25, adding to 5, which is N, so its standard deviation is \displaystyle \sqrt{\frac{5}{5}} = 1. Figure 1 shows the two steps: the first slides the dots so the mean sits at 0, the second relabels the axis in steps of 6 meters.
A sample, and a skew that stays
A baker timed how long five batches of bread dough took to rise, as a sample of all her batches: 26, 28, 28, 32 and 36 minutes. Standardize the times and describe the shape before and after.
SolutionThese five batches stand in for all the batches she makes, so they are a sample: use \bar{x} and s, dividing by n - 1 = 4.
The mean is \displaystyle \bar{x} = \frac{150}{5} = 30 minutes, and the deviations are -4, -2, -2, 2, 6. Their squares add to 16 + 4 + 4 + 4 + 36 = 64, so
Dividing each deviation by 4 gives the standardized times
Check. They add to 0, and their squares, 1 + 0.25 + 0.25 + 0.25 + 2.25 = 4, add to n - 1 = 4, as a sample’s must.
The raw times bunch between 26 and 28 and trail off to 36: skewed to the right. The standardized times bunch between -1 and -0.5 and trail off to 1.5, the same shape exactly (Figure 2). Standardizing put the center at 0 and the spread at 1 and did nothing else.
Two judges on one scale
Two judges each marked the same five science fair projects, A to E, out of 10.
| project | first judge | second judge |
|---|---|---|
| A | 9 | 6 |
| B | 7 | 10 |
| C | 7 | 8 |
| D | 7 | 2 |
| E | 5 | 4 |
Adding the raw marks puts B first. Standardize each judge’s marks, add the two standardized marks for each project, and say which project comes first now, and why.
SolutionEach judge’s five marks are every mark that judge gave in this contest, so treat each list as a population.
First judge: the mean is \displaystyle \frac{35}{5} = 7, the deviations are 2, 0, 0, 0, -2, their squares add to 8, and \displaystyle \sigma_1 = \sqrt{\frac{8}{5}} = \sqrt{1.6} \approx 1.265. The standardized marks are \displaystyle \frac{2}{\sqrt{1.6}} = \sqrt{2.5} \approx 1.58 for A, 0 for B, C and D, and -1.58 for E.
Second judge: the mean is \displaystyle \frac{30}{5} = 6, the deviations are 0, 4, 2, -4, -2, their squares add to 40, and \displaystyle \sigma_2 = \sqrt{\frac{40}{5}} = \sqrt{8} \approx 2.828. The standardized marks are 0 for A, \displaystyle \frac{4}{\sqrt{8}} = \sqrt{2} \approx 1.41 for B, 0.71 for C, -1.41 for D and -0.71 for E.
| project | raw total | standardized total |
|---|---|---|
| A | 15 | 1.58 |
| B | 17 | 1.41 |
| C | 15 | 0.71 |
| D | 9 | -1.41 |
| E | 9 | -2.29 |
On the standardized scale A comes first and B second. The reason is the spreads. The second judge’s marks have more than twice the standard deviation of the first judge’s, so in the raw total the second judge’s opinion counts for more: a mark of 10 from a judge who ranges from 2 to 10 moves the total further than a 9 from a judge who stays between 5 and 9. A’s 9 was the first judge’s clear top mark, 1.58 standard deviations up, while B’s 10, though higher on paper, was 1.41 standard deviations up for the second judge. Standardizing gives each judge the same weight. Whether equal weight is what the contest wants is a decision for its organizers; the mathematics only shows what each choice does.
Getting the original values back
A music app shows the length of each of the five tracks on a short album only as a standardized value. The album’s tracks have mean length 140 seconds and standard deviation 20 seconds, taken as a population. Four of the standardized values are -1.5, -0.7, 0.9 and 1.2, and the fifth is missing. Find it, check it, and find all five track lengths.
SolutionStandardized values always add to 0. The four known ones add to -1.5 - 0.7 + 0.9 + 1.2 = -0.1, so the fifth is 0.1.
Check with the second fact: for a population of 5 the squares must add to 5.
They do, so the five values are consistent with a population of five tracks. Now undo the standardizing with x = \mu + z\sigma = 140 + 20z:
| z | length (s) |
|---|---|
| -1.5 | 110 |
| -0.7 | 126 |
| 0.1 | 142 |
| 0.9 | 158 |
| 1.2 | 164 |
As a final check, these lengths add to 700, so their mean is \displaystyle \frac{700}{5} = 140; the deviations -30, -14, 2, 18, 24 have squares adding to 2000, and \displaystyle \sqrt{\frac{2000}{5}} = \sqrt{400} = 20.
A change of unit leaves the standardized values alone
Take the rising times of Example 2, 26, 28, 28, 32, 36 minutes. (a) Write them in seconds and standardize again. (b) The baker’s schedule allows 40 minutes for rising, so each batch has 40 - x minutes to spare. Standardize the minutes to spare. Explain both results.
Solution(a) In seconds the times are 1560, 1680, 1680, 1920, 2160. Multiplying by 60 multiplies the mean and the sample standard deviation by 60: \bar{x} = 1800 seconds and s = 240 seconds. The first standardized value is \displaystyle \frac{1560 - 1800}{240} = \frac{-240}{240} = -1, and the others work out the same way: -1, -0.5, -0.5, 0.5, 1.5, exactly as in minutes.
This happens for every change y = a + bx with b > 0. The mean becomes a + b\bar{x} and the standard deviation bs, so
The a cancels in the subtraction and the b in the division. Standardized values do not know what unit the data were measured in, which is why they can be compared across units.
(b) The minutes to spare are 14, 12, 12, 8, 4, with mean 40 - 30 = 10. Here b = -1, and a negative multiplier multiplies the standard deviation by |b| = 1, so it stays 4. The standardized values are
the old ones with every sign changed. With b < 0 the same algebra gives \displaystyle \frac{b(x - \bar{x})}{|b|s} = -\frac{x - \bar{x}}{s}, so the order reverses: the slowest batch had the least time to spare. A right skew in rising times became a left skew in time to spare.
How far from the mean can a value be?
Show that in a population of N values (N \ge 2, not all equal), no standardized value is larger than \displaystyle \sqrt{N - 1} in size. What does this say for a population of 5, and which data reach the limit?
SolutionPick any one standardized value and call it z_1; call the other N - 1 values z_2, \ldots, z_N. The two free checks say all N add to 0 and their squares add to N, so for the others
For any k numbers w_1, \ldots, w_k with mean \bar{w}, the sum of squares is at least the square of the sum divided by k, because the difference is a sum of squared deviations, which is never negative:
Apply it to the N - 1 other values:
Multiply both sides by the positive number N - 1 and collect the z_1^2 terms:
So \displaystyle |z_1| \le \sqrt{N - 1}. Equality needs \sum (w - \bar{w})^2 = 0, that is, the other N - 1 values all equal.
For N = 5 the limit is \displaystyle \sqrt{4} = 2: in a population of five, no value is more than two standard deviations from the mean. The data 10, 10, 10, 10, 20 reach it. Their mean is 12, the deviations are -2, -2, -2, -2, 8, the squares add to 80, and \displaystyle \sigma = \sqrt{16} = 4, so the standardized values are -0.5, -0.5, -0.5, -0.5, 2. In a small data set a large standardized value is impossible, not merely rare, which is worth knowing before calling a value in a small set “unusual”.
Boundaries
The standard deviation must not be 0. If every value is the same, \sigma = 0 and the division is undefined. There is no spread to measure in, and the set cannot be standardized.
One mean and one standard deviation, from the same set. Each value is standardized with the mean and standard deviation of the set it belongs to. Values from different sets are put on a common scale by standardizing each set with its own summaries, as the two judges were in Example 3.
Population or sample changes the numbers. Using s instead of \sigma on the same data divides by a larger number, since \displaystyle s = \sigma\sqrt{\frac{n}{n - 1}}, so every standardized value is multiplied by \displaystyle \sqrt{\frac{n - 1}{n}} and lies a little closer to 0. The new set has mean 0 either way, and standard deviation 1 in whichever sense was used to standardize. For the rising times of Example 2, the population standard deviation is \displaystyle \sqrt{12.8} \approx 3.58 minutes, which would give -1.12 rather than -1 for the quickest batch. Say which one you used.
The shape is untouched. Standardizing changes only the numbers on the axis. A skewed set stays skewed, a set with two peaks keeps two peaks, and an outlier stays an outlier. The result is called standard normal only when the data were normal to begin with.
A value’s meaning depends on the shape. “One and a half standard deviations above the mean” is exact for any data, but how unusual that is depends on the shape and on the size of the set. In a set of N values no standardized value can exceed \displaystyle \sqrt{N - 1} (Example 6).
Only shifting and positive rescaling leave the standardized values alone. A change of unit or any y = a + bx with b > 0 gives back the same standardized values; b < 0 flips every sign. A change that is not of this form, such as squaring every value, changes them in ways that must be worked out again from the data.
Keep the mean and standard deviation exact. Rounding them before dividing makes every standardized value slightly wrong; round only at the end.
Where this goes next
Reading single standardized values, and what a value of 2 or -3 means for data shaped like a bell, is the work of z-scores and the empirical rule, which adds the empirical rule. When the data follow a normal distribution, standardizing turns every normal curve into one curve, the standard normal distribution, with mean 0 and standard deviation 1, so a single table or calculator function covers them all; areas under it are read as in Density curves: area, mean and median.
Standardizing two variables at once leads to correlation: the correlation coefficient is, up to the factor that comes from dividing by n - 1, the average product of the two standardized values of each pair. Because standardized values have no unit, the correlation has none either, and a change of unit cannot change it. The habit of measuring a distance in standard deviations, rather than in the data’s own unit, runs through the rest of statistics, and reading a standard deviation in its setting is in Interpreting a standard deviation.
Exercises
Standardizing a whole set
- Standardize the population 2, 4, 4, 4, 5, 5, 7, 9.
- Standardize the population 46, 46, 50, 52, 53, 53. Give exact answers.
- A sample of six delivery times, in minutes, is 22, 24, 25, 25, 26, 28. Standardize the times as a sample.
- Standardize the six times of exercise 3 again, this time treating them as a population. Give each to two decimal places and compare with exercise 3.
Checking and undoing
- A population of four values has mean 50 and standard deviation 5. Its standardized values are -1.4, -0.2, 0.2, 1.4. Find the original values, and check that the squares of the standardized values add to the right number.
- A population of five values was standardized using mean 200 and standard deviation 10. Four of the standardized values are -1.3, -0.9, 0.9 and 1.3. Find the fifth, check it, and find all five original values.
- Could -1.2, 0.3, 0.5, 0.9 be the standardized values of a data set? Explain.
- Could -1, -1, 0, 1, 1 be the standardized values of a population of five? Of a sample of five?
Units and rules
- The values in exercise 5 are lengths in centimeters. Write them in millimeters and standardize them again.
- Each value x of the population in exercise 1 is changed to 20 - 2x. Standardize the new values, and say how they compare with the answer to exercise 1.
- A report puts every data set on a scale with mean 50 and standard deviation 10, using T = 50 + 10z. Find T for each hole of the disc golf course in Example 1.
- Find the rule y = a + bx that turns the hole lengths of Example 1 into a set with mean 100 and standard deviation 15, and give the five new values.
Summaries after standardizing
- A population has mean 40, median 37, standard deviation 8, Q_1 = 33 and Q_3 = 46. Find the median, the quartiles and the interquartile range of the standardized data.
- A population of 40 values has mean 120, standard deviation 15, minimum 90 and maximum 165. Find the minimum, the maximum and the range of the standardized data.
How far a value can be
- What is the largest standardized value possible in a population of four values? Give a data set that reaches it.
- Someone reports a standardized value of 3.5 in a population of ten values. Explain why this must be a mistake.
Reasoning
- Explain why a standardized value has no unit, and why that matters when comparing data sets.
- Prove that the standardized values of any population of N values add to 0 and that their squares add to N.
- A data set is strongly skewed to the right. Someone says that standardizing it will make it symmetric, with mean 0. Which part of that is true, and which is not? Explain.
- Explain why standardizing subtracts the mean first and divides by the standard deviation second, and what goes wrong in the other order.
- Explain why the data 8, 8, 8, 8 cannot be standardized, in terms of what a standardized value measures.
Answers
- -1.5, -0.5, -0.5, -0.5, 0, 0, 1, 2. The mean is \displaystyle \frac{40}{8} = 5, the deviations are -3, -1, -1, -1, 0, 0, 2, 4, their squares add to 32, and \displaystyle \sigma = \sqrt{\frac{32}{8}} = 2.
- \displaystyle -\frac{4}{3}, -\frac{4}{3}, 0, \frac{2}{3}, 1, 1. The mean is \displaystyle \frac{300}{6} = 50, the deviations -4, -4, 0, 2, 3, 3 have squares adding to 54, and \displaystyle \sigma = \sqrt{\frac{54}{6}} = 3.
- -1.5, -0.5, 0, 0, 0.5, 1.5. The mean is \displaystyle \frac{150}{6} = 25, the deviations -3, -1, 0, 0, 1, 3 have squares adding to 20, and \displaystyle s = \sqrt{\frac{20}{5}} = 2. Check: the squares of the answers add to 5 = n - 1.
- -1.64, -0.55, 0, 0, 0.55, 1.64. Now \displaystyle \sigma = \sqrt{\frac{20}{6}} \approx 1.83, smaller than s = 2, so each value is about 1.1 times as far from 0 as in exercise 3: every one is multiplied by \displaystyle \sqrt{\frac{6}{5}}.
- 43, 49, 51, 57, from x = 50 + 5z. The squares 1.96 + 0.04 + 0.04 + 1.96 = 4 add to N = 4.
- The fifth is 0, since the four add to 0 already. Check: 1.69 + 0.81 + 0 + 0.81 + 1.69 = 5. The values are 187, 191, 200, 209, 213, from x = 200 + 10z.
- No. They add to 0.5, and standardized values always add to 0.
- Not of a population: the squares add to 4, and a population of five needs 5. Yes of a sample: a sample of five needs n - 1 = 4, and the values also add to 0.
- 430, 490, 510, 570 mm have mean 500 mm and standard deviation 50 mm, so the standardized values are -1.4, -0.2, 0.2, 1.4, unchanged: a change of unit multiplies the deviations and the standard deviation by the same 10.
- 1.5, 0.5, 0.5, 0.5, 0, 0, -1, -2, the answers to exercise 1 with every sign changed. The new values 16, 12, 12, 12, 10, 10, 6, 2 have mean 20 - 2 \times 5 = 10 and standard deviation |-2| \times 2 = 4, and the multiplier -2 is negative.
- 35, 45, 50, 55, 65, from T = 50 + 10z with z = -1.5, -0.5, 0, 0.5, 1.5.
- y = 2.5x - 52.5, giving 77.5, 92.5, 100, 107.5, 122.5. The rule is \displaystyle y = 100 + 15 \cdot \frac{x - 61}{6} = 100 + 2.5x - 152.5.
- Median -0.375, Q_1 = -0.875, Q_3 = 0.75, IQR 1.625. Each is (value - 40) \div\, 8: \displaystyle \frac{-3}{8}, \displaystyle \frac{-7}{8}, \displaystyle \frac{6}{8}; the IQR is \displaystyle \frac{13}{8}, the old IQR of 13 divided by 8. The median is still below the mean, as it was before.
- Minimum -2, maximum 3, range 5. \displaystyle \frac{90 - 120}{15} = -2, \displaystyle \frac{165 - 120}{15} = 3, and the old range 75 divided by 15 is 5.
- \displaystyle \sqrt{3} \approx 1.73, by Example 6 with N = 4. The data 0, 0, 0, 4 reach it: mean 1, deviations -1, -1, -1, 3, \displaystyle \sigma = \sqrt{\frac{12}{4}} = \sqrt{3}, and \displaystyle z = \frac{3}{\sqrt{3}} = \sqrt{3}.
- In a population of ten, no standardized value can exceed \displaystyle \sqrt{10 - 1} = 3 in size (Example 6), and 3.5 > 3.
- The numerator x - \mu and the denominator \sigma are both in the data’s unit, so the unit cancels. That lets values measured in meters, minutes or marks be placed on one scale, and it is why a change of unit leaves standardized values unchanged.
- \displaystyle \sum z = \frac{1}{\sigma}\sum (x - \mu) = \frac{1}{\sigma}\left(\sum x - N\mu\right) = 0, because \sum x = N\mu. And \displaystyle \sum z^2 = \frac{1}{\sigma^2}\sum (x - \mu)^2 = \frac{N\sigma^2}{\sigma^2} = N, because \displaystyle \sigma^2 = \frac{1}{N}\sum (x - \mu)^2.
- The mean 0 is true, and so is a standard deviation of 1. “Symmetric” is false: standardizing is a shift and a positive rescale, which keep the order and the gaps in proportion, so the long right tail is still there and the set is still skewed to the right.
- Subtracting \mu moves the mean to 0 and leaves the spread at \sigma; dividing by \sigma then keeps the mean at 0 and brings the spread to 1. In the other order, dividing first makes the mean \displaystyle \frac{\mu}{\sigma}, so subtracting \mu leaves mean \displaystyle \frac{\mu}{\sigma} - \mu, which is 0 only when \mu = 0 or \sigma = 1. The spread is 1 either way.
- A standardized value says how many standard deviations a value is from the mean. Here every value equals the mean, 8, so the standard deviation is 0: there is no unit of spread to count in, and \displaystyle \frac{8 - 8}{0} is undefined.