Statistics

Variance and standard deviation of a random variable

Two random variables can have the same mean and still behave very differently. How do you measure how far a random variable’s values typically land from its mean?

Unlocks

Mean and standard deviation of a binomialStatistics

Where you are 11 in Random variables

Probability4

Random variables11

▴ 6 above Random variables and probability distributions The binomial distribution4 The geometric distribution

Variance and standard deviation of a random variable

Mean and standard deviation of a geometric random variable

When you reach for it

A Random variables and probability distributions lists the values of a random variable X and their probabilities, and the mean \mu_X = \sum x \, P(x) says where X balances in the long run. The mean says nothing about spread. A player who always takes 3 strokes on a hole and a player who swings between a hole in one and 5 strokes can share the mean 3. To tell them apart you need a measure of how far the values typically fall from \mu_X: the variance \sigma_X^2 and its square root, the standard deviation \sigma_X. You reach for them whenever the question is about consistency, risk, or how much one result may differ from what is expected.

All the settings below are made up: a mini golf course run by Josu, where Oihane and Garazi play.

The procedure

Day 1: the variance and standard deviation from a table

Start from the distribution and its mean. For each value x, the deviation x - \mu_X says how far that value is from the mean and on which side. The deviations cannot simply be averaged: weighted by probability they always add to zero, because \sum (x - \mu_X) P(x) = \sum x P(x) - \mu_X \sum P(x), which is \mu_X - \mu_X = 0. Squaring each deviation makes every distance count as positive, exactly as for data in Computing a standard deviation, sample and population. Then each squared deviation is weighted by how likely its value is, and the results are added:

\sigma_X^2 = \operatorname{Var}(X) = \sum (x - \mu_X)^2 \, P(x)

The standard deviation is the square root, which brings the measure back to the unit of X:

\sigma_X = \sqrt{\operatorname{Var}(X)}

Why weighting by probability is the right average. Suppose Oihane’s strokes on a hole are 2, 3 or 4 with probabilities 0.25, 0.5 and 0.25, so \mu_X = 3. Picture 100 rounds that follow those probabilities exactly: 25 twos, 50 threes and 25 fours. The variance of those 100 numbers, adding the squared deviations and dividing by 100, is

\begin{aligned} &\frac{25(2-3)^2 + 50(3-3)^2 + 25(4-3)^2}{100} \\ &= (2-3)^2(0.25) + (3-3)^2(0.5) \\ &\quad + (4-3)^2(0.25) \end{aligned}

Dividing each count by 100 turns it into a probability, so averaging over the long run and weighting by probability are the same calculation.

Why there is no n - 1. A data standard deviation s_x divides by n - 1 because the sample mean \bar{x} is estimated from the same few values, which makes their squared deviations run a little small. A random variable’s distribution is a model of the whole long run, and \mu_X is its exact mean, not an estimate from a sample. Nothing is estimated, so nothing needs correcting: the probabilities do the dividing. The formula matches the population formula, the one that divides by N.

Units and meaning. The variance is in squared units (square strokes, square dollars), which is hard to read. The standard deviation is in the unit of X itself, and it reads as a typical distance: in the long run, a value of X typically lands about \sigma_X away from \mu_X. As with data (Interpreting a standard deviation), it is never larger than the largest distance of a possible value from the mean, and it is 0 only when one value has probability 1.

Figure 1Same mean, same probabilities, but Garazi’s values sit twice as far from 3, so her standard deviation is twice Oihane’s (made-up probabilities).

Day 2: a shortcut, comparing variables, and the long run

The shortcut. Expanding the square gives a second formula. Write \mu for \mu_X:

\begin{aligned} \sum (x - \mu)^2 P(x) &= \sum x^2 P(x) - 2\mu \sum x P(x) \\ &\quad + \mu^2 \sum P(x) \\ &= E(X^2) - 2\mu \cdot \mu + \mu^2 \cdot 1 \\ &= E(X^2) - \mu_X^2 \end{aligned}

The first line splits (x - \mu)^2 = x^2 - 2\mu x + \mu^2 and adds term by term; the second uses \sum x P(x) = \mu and \sum P(x) = 1. Here E(X^2) = \sum x^2 P(x) is the probability-weighted mean of the squared values. So

\operatorname{Var}(X) = E(X^2) - \mu_X^2

The mean of the squares minus the square of the mean. It skips the deviation column, but it subtracts two numbers that can be close, so keep \mu_X exact: a rounded mean can throw the answer off badly (Example 5).

Comparing variables with equal means. When two random variables share a mean, the standard deviation separates them. The smaller one is the more consistent, or less risky, variable; the larger one gives both better and worse results more often. Which is preferable depends on the goal, and the probability of the interval that matters (Cumulative probability) settles it.

The long run. If the situation is repeated many times, or simulated, the results are data, and their standard deviation s_x (with n - 1, since the results are a sample of the long run) tends to settle near \sigma_X, just as their mean \bar{x} settles near \mu_X. A short run can miss by a lot; a long run usually misses by little. With many trials, dividing by n or by n - 1 makes almost no difference.

Adding a constant to every value or multiplying every value by a constant changes \sigma_X in a simple way, the subject of Linear transformations of a random variable; the spread of a sum of two random variables is Combining random variables.

Worked examples

Example 1

Day 1: two players, one mean

On hole 7, Oihane’s strokes X and Garazi’s strokes Y have these distributions (made-up probabilities). Find the mean, variance and standard deviation of each, and compare.

strokes x	2	3	4
P(x)	0.25	0.5	0.25

strokes y	1	3	5
P(y)	0.25	0.5	0.25

Solution

The means: \mu_X = 2(0.25) + 3(0.5) + 4(0.25) = 3 and \mu_Y = 1(0.25) + 3(0.5) + 5(0.25) = 3 strokes. Both players average 3 strokes in the long run.

For X, the deviations from 3 are -1, 0, 1, so

\begin{aligned} \sigma_X^2 &= (-1)^2(0.25) + 0^2(0.5) + 1^2(0.25) \\ &= 0.5 \end{aligned}

square strokes, and \displaystyle \sigma_X = \sqrt{0.5} \approx 0.71 strokes. For Y, the deviations are -2, 0, 2, so

\begin{aligned} \sigma_Y^2 &= (-2)^2(0.25) + 0^2(0.5) + 2^2(0.25) \\ &= 2 \end{aligned}

square strokes, and \displaystyle \sigma_Y = \sqrt{2} \approx 1.41 strokes.

Garazi’s outer values sit twice as far from the mean as Oihane’s. Squaring makes her variance 4 times as large, and the square root brings the factor back to 2: \sigma_Y = 2\sigma_X. In the long run Oihane’s score on this hole is typically about 0.71 strokes from 3, Garazi’s about 1.41 strokes from 3. Figure 1 shows both.

Example 2

Day 1: a mean that is not a whole number

When a group arrives at the first tee, let X be the number of groups already waiting. Josu’s model (made-up probabilities):

groups waiting x	0	1	2	3
P(x)	0.1	0.3	0.4	0.2

Find \sigma_X and interpret it.

Solution

The mean is \mu_X = 0(0.1) + 1(0.3) + 2(0.4) + 3(0.2) = 1.7 groups, kept exact. A column for each step, writing d = x - 1.7 for the deviation:

x	P(x)	d	d^2	d^2 P(x)
0	0.1	-1.7	2.89	0.289
1	0.3	-0.7	0.49	0.147
2	0.4	0.3	0.09	0.036
3	0.2	1.3	1.69	0.338
sum	1			0.81

So \sigma_X^2 = 0.81 square groups and \displaystyle \sigma_X = \sqrt{0.81} = 0.9 groups. Over many arrivals, the number of groups waiting averages 1.7 and typically differs from 1.7 by about 0.9 of a group. It is sensible: the distances from the mean run from 0.3 to 1.7, and 0.9 lies between them.

Example 3

Day 1: a missing probability and a rare large value

Josu’s “lucky ball” game gives each player one free draw from a bucket. The prize X, in dollars, is 0 with probability 0.7, 5 with probability 0.25, and 20 otherwise (made-up). Find the standard deviation of the prize, give the units of the variance, and interpret \sigma_X.

Solution

The probabilities must add to 1, so P(20) = 1 - 0.7 - 0.25 = 0.05. The mean is \mu_X = 0(0.7) + 5(0.25) + 20(0.05) = 2.25 dollars. The deviations d = x - 2.25 are -2.25, 2.75 and 17.75.

x	P(x)	d^2	d^2 P(x)
0	0.7	5.0625	3.54375
5	0.25	7.5625	1.890625
20	0.05	315.0625	15.753125
sum	1		21.1875

The variance is \sigma_X^2 = 21.1875 square dollars, a unit with no everyday meaning. The standard deviation is \displaystyle \sigma_X = \sqrt{21.1875} \approx 4.60 dollars. Over many draws the prize averages 2.25 dollars and typically differs from that by about 4.60 dollars.

The standard deviation is larger than the mean, and the last row explains why: the rare 20-dollar prize supplies 15.753125 of the 21.1875, about three quarters, because squaring weighs a far value heavily even when it is unlikely.

Example 4

Day 2: the shortcut, checked against the definition

On the windmill hole, Josu’s strokes X are 2, 3, 4 or 5 with probabilities 0.1, 0.2, 0.3 and 0.4 (made-up). Find \sigma_X with the shortcut, then check with the definition.

Solution

The mean is \mu_X = 2(0.1) + 3(0.2) + 4(0.3) + 5(0.4) = 4 strokes. The mean of the squares weights each squared value by its probability:

\begin{aligned} E(X^2) &= 4(0.1) + 9(0.2) + 16(0.3) + 25(0.4) \\ &= 17 \end{aligned}

So \operatorname{Var}(X) = 17 - 4^2 = 1 square stroke and \sigma_X = 1 stroke.

Check: the deviations from 4 are -2, -1, 0, 1, and (-2)^2(0.1) + (-1)^2(0.2) + 0^2(0.3) + 1^2(0.4) is 0.4 + 0.2 + 0 + 0.4 = 1. The two formulas agree, as the algebra says they must.

Example 5

Day 2: keep the mean exact

Let X be the number of golf balls Josu fishes out of the course’s pond at closing: 1, 2 or 5 with probabilities 0.5, 0.3 and 0.2 (made-up). Find \sigma_X with the shortcut, and see what a rounded mean would do.

Solution

\mu_X = 1(0.5) + 2(0.3) + 5(0.2) = 2.1 balls, and

E(X^2) = 1(0.5) + 4(0.3) + 25(0.2) = 6.7

So \operatorname{Var}(X) = 6.7 - 2.1^2 = 6.7 - 4.41 = 2.29 square balls, and \displaystyle \sigma_X = \sqrt{2.29} \approx 1.51 balls. The definition agrees: (-1.1)^2(0.5) + (-0.1)^2(0.3) + 2.9^2(0.2) is 0.605 + 0.003 + 1.682 = 2.29.

Had the mean been rounded to 2, the shortcut would give 6.7 - 4 = 2.7 and \displaystyle \sqrt{2.7} \approx 1.64, wrong in the first decimal place. The rounding error of 0.1 in \mu_X became an error of 0.41 in the variance, because \mu_X^2 is subtracted from a number not much larger than itself.

Example 6

Day 2: same mean, different risk

In a team final, the last hole is worth 3 strokes on average for either player. Oihane’s strokes X are 2, 3, 4 with probabilities 0.2, 0.6, 0.2. Garazi’s strokes Y are 1, 2, 4, 5, each with probability 0.25 (made-up). (a) Find both standard deviations. (b) The team wins only with a hole in one. Who should play? (c) The team wins with 3 strokes or fewer. Who should play?

Solution

(a) Both means are 3: \mu_X = 0.4 + 1.8 + 0.8 = 3 and \mu_Y = 0.25(1 + 2 + 4 + 5) = 3. Then

\begin{aligned} \sigma_X^2 &= (-1)^2(0.2) + 0^2(0.6) + 1^2(0.2) \\ &= 0.4, \quad \sigma_X \approx 0.63 \end{aligned}

\begin{aligned} \sigma_Y^2 &= 0.25\big((-2)^2 + (-1)^2 + 1^2 + 2^2\big) \\ &= 2.5, \quad \sigma_Y \approx 1.58 \end{aligned}

in strokes. Oihane is the consistent player; Garazi’s results spread about two and a half times as far.

(b) A hole in one is a result far from the mean, which only the spread-out player ever reaches: P(X = 1) = 0 but P(Y = 1) = 0.25. Garazi should play.

(c) Now the team needs to avoid the high end. P(X \le 3) = 0.2 + 0.6 = 0.8, while P(Y \le 3) = 0.25 + 0.25 = 0.5. Oihane should play. The means could not decide either question; the spread, read through the probability of the interval that matters, decides both.

Example 7

Day 2: the long run approaches \sigma

Josu records Garazi’s strokes on hole 7, the variable Y of Example 1, with \sigma_Y \approx 1.41. Her first 20 rounds give 1 stroke 7 times, 3 strokes 9 times and 5 strokes 4 times. Over the whole season of 400 rounds the counts are 96, 207 and 97 (made-up results). Find the mean and standard deviation of each set of results, and compare with the model.

Solution

These are data, a sample of Garazi’s long run, so they get \bar{x} and s_x with n - 1.

First 20 rounds: \displaystyle \bar{x} = \frac{1(7) + 3(9) + 5(4)}{20}, which is \displaystyle \frac{54}{20} = 2.7 strokes. The squared deviations from 2.7 are 2.89, 0.09 and 5.29, so the sum of squares is 7(2.89) + 9(0.09) + 4(5.29) = 42.2, and \displaystyle s_x = \sqrt{\frac{42.2}{19}} \approx 1.49 strokes.

Season of 400 rounds: \displaystyle \bar{x} = \frac{1(96) + 3(207) + 5(97)}{400}, which is \displaystyle \frac{1202}{400} = 3.005 strokes. The sum of squared deviations is 771.99, so \displaystyle s_x = \sqrt{\frac{771.99}{399}} \approx 1.39 strokes. Dividing by 400 instead gives 1.389, the same to two decimal places.

The model says \mu_Y = 3 and \sigma_Y \approx 1.41. After 20 rounds the mean is off by 0.3 and the standard deviation by about 0.08; after 400 rounds by 0.005 and about 0.02. The results settle toward the model as rounds pile up, which is what “in the long run” means in the interpretation of \sigma_Y. They never have to match it exactly.

Boundaries

The values must be numbers on a scale where distance means something. A coded category (1 for red, 2 for blue) has no meaningful deviation.
The probabilities must be between 0 and 1 and add to 1; otherwise the table is not a distribution and the weighted sum measures nothing.
These formulas are for a discrete variable with a list of values. For a continuous variable the sum becomes an area under a density curve. A variable with infinitely many values may have an infinite sum, and then it has no variance.
\sigma_X describes the model. Real results match it only approximately, and only if the model is right; a run of trials estimates it (Example 7).
The standard deviation measures spread about the mean and nothing else. Two distributions can share \mu_X and \sigma_X and still differ in shape.
In the shortcut, keep \mu_X exact; rounding it can move the variance a long way (Example 5).

Where this goes next

The mean and standard deviation together describe a random variable’s center and spread, the pair used for every distribution that follows. Linear transformations of a random variable shows how a change of units or a fixed fee moves them, and Combining random variables shows why spreads combine through variances, not standard deviations. The Mean and standard deviation of a binomial gives \mu_X and \sigma_X for a count of successes by formula, and a normal random variable is described completely by these two numbers.

Exercises

Day 1: variance and standard deviation from a table

All probabilities are made up. Give \sigma_X to two decimal places unless it is exact.

The number of balls a group loses in the pond is 0, 1 or 2 with probabilities 0.5, 0.3 and 0.2. Find \operatorname{Var}(X) and \sigma_X.
A group booking at the course has 1, 2, 3 or 4 players, each with probability 0.25. Find \sigma_X.
A scratch card at the snack bar wins 0 dollars with probability 0.8 and 10 dollars with probability 0.2. Find \mu_X, the variance with its units, and \sigma_X.
X takes the values 2, 4, 6, 8 with P(2) = 0.1, P(4) = 0.4, P(8) = 0.2. Find P(6), then \mu_X and \sigma_X.
On hole 12, Oihane’s strokes have \mu_X = 3.4 and \sigma_X = 0.8 strokes. Write one sentence interpreting each number.
X takes the values 1, 2, 3 with probabilities 0.2, 0.5, 0.3. Explain without computing why \sigma_X = 3 is impossible, then find \sigma_X.
(a) X is 0 or 10, each with probability 0.5. Find \sigma_X. (b) Y is 5 with probability 1. Find \sigma_Y.
X takes the values 2, 4, 6 with probabilities 0.2, 0.6, 0.2. Garazi adds the three squared deviations and divides by 3 - 1 = 2, getting a standard deviation of 2. Find the correct \sigma_X, and say what went wrong.

Day 2: the shortcut, comparisons and the long run

X takes the values 0, 1, 2, 3 with probabilities 0.4, 0.3, 0.2, 0.1. Use the shortcut to find \sigma_X.
X takes the values 1, 4, 9 with probabilities 0.5, 0.25, 0.25. Use the shortcut to find \operatorname{Var}(X) and \sigma_X.
X takes the values 2 and 5 with probabilities 0.6 and 0.4. Find \operatorname{Var}(X) both by the definition and by the shortcut.
(a) A random variable has \mu_X = 2.5 and E(X^2) = 7.25. Find \sigma_X. (b) Explain why no random variable has \mu_X = 2.5 and E(X^2) = 5.
On the last hole, player A takes 2, 3 or 4 strokes with probabilities 0.3, 0.4, 0.3; player B takes 1 or 5 strokes, each with probability 0.5. (a) Find both means and standard deviations. (b) Who is more consistent? (c) Who gives the better chance of a hole in one, and who of 4 strokes or fewer?
For a charity night Josu can choose plan A, which raises 300 dollars for certain, or plan B, which raises 0 dollars with probability 0.4 and 500 dollars with probability 0.6. Find the mean and standard deviation of each, and describe the choice.
Josu records the balls lost by 50 groups whose losses follow Exercise 1’s distribution: 26 lose 0, 14 lose 1 and 10 lose 2. Find \bar{x} and s_x, and compare them with \mu_X and \sigma_X from Exercise 1.
Josu says that if he records 10{,}000 rounds, the standard deviation of the results will equal \sigma_X exactly. Is he right? Explain.

Reasoning

Explain why \operatorname{Var}(X) \ge 0 for every random variable, and why it equals 0 only when one value has probability 1.
Explain why \sigma_X is never larger than the largest distance |x - \mu_X| of a possible value from the mean.
Explain why the variance of a random variable has no n - 1, while the standard deviation of a sample of data does.
Use the shortcut and Exercise 17 to show that E(X^2) \ge \mu_X^2 for every random variable. When are they equal?

Answers

\mu_X = 0.3 + 0.4 = 0.7. Squared deviations 0.49, 0.09, 1.69, weighted 0.245, 0.027, 0.338. \operatorname{Var}(X) = 0.61 square balls, \displaystyle \sigma_X = \sqrt{0.61} \approx 0.78 balls.
\mu_X = 2.5. Squared deviations 2.25, 0.25, 0.25, 2.25, each times 0.25, sum 1.25. \displaystyle \sigma_X = \sqrt{1.25} \approx 1.12 players.
\mu_X = 10(0.2) = 2 dollars. \operatorname{Var}(X) = (-2)^2(0.8) + 8^2(0.2) = 3.2 + 12.8 = 16 square dollars, \sigma_X = 4 dollars.
P(6) = 1 - 0.1 - 0.4 - 0.2 = 0.3. \mu_X = 0.2 + 1.6 + 1.8 + 1.6 = 5.2. Weighted squared deviations 1.024, 0.576, 0.192, 1.568, sum 3.36. \displaystyle \sigma_X = \sqrt{3.36} \approx 1.83.
Over many rounds Oihane averages 3.4 strokes on hole 12. Her score on that hole is typically about 0.8 strokes away from 3.4.
The mean lies between the smallest and largest values, 1 and 3, so no value is more than 2 from it, and \sigma_X is never larger than the largest distance (Exercise 18): \sigma_X \le 2. Computing: \mu_X = 0.2 + 1 + 0.9 = 2.1, and the distances from it are 1.1, 0.1, 0.9, so \sigma_X is in fact at most 1.1. Weighted squared deviations 0.242, 0.005, 0.243, sum 0.49, so \displaystyle \sigma_X = \sqrt{0.49} = 0.7 exactly.
(a) \mu_X = 5, \operatorname{Var}(X) = 25(0.5) + 25(0.5) = 25, \sigma_X = 5. (b) Every result is the mean, so \sigma_Y = 0.
\mu_X = 0.4 + 2.4 + 1.2 = 4. \operatorname{Var}(X) = 4(0.2) + 0(0.6) + 4(0.2) = 1.6, so \displaystyle \sigma_X = \sqrt{1.6} \approx 1.26. Garazi treated the three values as a sample of three data points, ignoring the probabilities (4 is three times as likely as either end) and dividing by n - 1, which a random variable does not use.
\mu_X = 0.3 + 0.4 + 0.3 = 1. E(X^2) = 0.3 + 0.8 + 0.9 = 2. \operatorname{Var}(X) = 2 - 1 = 1, \sigma_X = 1.
\mu_X = 0.5 + 1 + 2.25 = 3.75. E(X^2) = 0.5 + 4 + 20.25 = 24.75. \operatorname{Var}(X) = 24.75 - 14.0625 = 10.6875, \sigma_X \approx 3.27.
\mu_X = 1.2 + 2 = 3.2. Definition: (-1.2)^2(0.6) + 1.8^2(0.4) = 0.864 + 1.296 = 2.16. Shortcut: E(X^2) = 2.4 + 10 = 12.4, and 12.4 - 3.2^2 = 12.4 - 10.24 = 2.16. Both give 2.16.
(a) \operatorname{Var}(X) = 7.25 - 2.5^2 = 7.25 - 6.25 = 1, so \sigma_X = 1. (b) The shortcut would give 5 - 6.25 = -1.25, a negative variance, which is impossible (a weighted sum of squares is never negative).
(a) Both means are 3. A: \operatorname{Var} = 0.3 + 0.3 = 0.6, \sigma \approx 0.77 strokes. B: \operatorname{Var} = 4(0.5) + 4(0.5) = 4, \sigma = 2 strokes. (b) A. (c) Hole in one: B, probability 0.5 against 0. Four or fewer: A, probability 1 against 0.5.
Both means are 300 dollars: B gives 0.6(500) = 300. Plan A has \sigma = 0. Plan B: \operatorname{Var} = 300^2(0.4) + 200^2(0.6), which is 36{,}000 + 24{,}000 = 60{,}000 square dollars, so \sigma \approx 244.95 dollars. Same long-run average; A is certain, B risks raising nothing in exchange for a chance at more.
\displaystyle \bar{x} = \frac{14 + 20}{50} = 0.68. Sum of squares 54, so the sum of squared deviations is 54 - 50(0.68)^2 = 30.88 and \displaystyle s_x = \sqrt{\frac{30.88}{49}} \approx 0.79. Close to \mu_X = 0.7 and \sigma_X \approx 0.78, but not equal, as expected from 50 groups.
No. The results are random, so their standard deviation is very likely close to \sigma_X after 10{,}000 rounds but almost never exactly equal to it. A longer run makes a large miss less likely; it does not remove the miss.
Each term (x - \mu_X)^2 P(x) is a square times a probability, so it is never negative, and the sum of such terms is never negative. The sum is 0 only if every term is 0, so every value with positive probability equals \mu_X. That leaves one value, which then has probability 1.
Let D be the largest distance. Every (x - \mu_X)^2 \le D^2, so \operatorname{Var}(X) \le D^2 \sum P(x) = D^2, and taking square roots keeps the order: \sigma_X \le D.
A sample’s mean \bar{x} is computed from the same values, which sit closer to their own mean than to the true one, so dividing by n - 1 corrects a sum that runs small. A random variable’s distribution is the whole long run, \mu_X is its exact mean, and the probabilities are the exact long-run shares, so nothing is estimated and there is nothing to correct.
E(X^2) - \mu_X^2 = \operatorname{Var}(X) \ge 0 by Exercise 17, so E(X^2) \ge \mu_X^2. They are equal exactly when the variance is 0, that is, when X takes one value with probability 1.