Statistics · S.ID.C.8

Correlation from standardized values

Technology reports a correlation r for a cloud of points. Where does that number come from, and why can it never pass 1 or -1?

You need

The correlation coefficientAlgebra 1 z-scores and the empirical ruleAlgebra 2

Where you are 3 in Correlation

Statistics7

Bivariate data9

Correlation3

Correlation from standardized values

Properties of the correlation coefficient The correlation coefficient

When you reach for it

The correlation coefficient r measures how tightly the points of a scatter plot cluster about a straight line, and technology reports it in one keystroke. This lesson opens the box. You reach for the formula when you want to compute r by hand for a small data set, when you want to see which points push r up and which pull it down, and when you want to know why r has no units, why it does not care which variable is called x, and why it never leaves the interval from -1 to 1. Each of those facts is visible in the formula and nowhere else.

The building block is the z-score: how many standard deviations a value sits from its mean. For paired data (x, y) each variable gets its own,

z_x = \frac{x - \bar{x}}{s_x}, \qquad z_y = \frac{y - \bar{y}}{s_y}.

The procedure

For n points, the correlation is

r = \frac{1}{n-1} \sum z_x z_y .

In words: standardize each variable, multiply the two z-scores of each point, add the products, and divide by n - 1. Each part has a reason.

Why z-scores. A z-score is a distance from the mean measured in standard deviations, so it has no units: centimeters divided by centimeters. Both variables now sit on one common scale with mean 0 and standard deviation 1, and a height and a weight can be multiplied without the answer depending on whether heights were in centimeters or inches.

Why products. The point (\bar{x}, \bar{y}) is the center of the cloud. Lines through it split the plot into four boxes (Figure 1). A point in the top-right box has both values above their means, so z_x > 0 and z_y > 0 and the product is positive. A point in the bottom-left box has both below, two negatives, and the product is again positive. A point in the top-left or bottom-right box has one z-score of each sign, and its product is negative. A point on either center line has a z-score of 0 and contributes nothing.

Figure 1Levan’s alpacas (made-up data). The dashed lines cross at the center of the cloud and split it into four boxes; the sign in each box is the sign of z_x z_y there.

So a cloud rising from left to right, which fills the top-right and bottom-left boxes, collects mostly positive products, and a falling cloud mostly negative ones. The size matters too: a point far from both center lines gives a large product, while a point near either line gives a small one, whatever happens in the other variable.

Why divide by n - 1. The sample standard deviation is defined by \displaystyle s_x^2 = \frac{1}{n-1}\sum (x - \bar{x})^2, so

\sum z_x^2 = \frac{\sum (x - \bar{x})^2}{s_x^2} = n - 1 .

The same holds for z_y. If every point lay on one rising line, each z_y would equal its z_x (Example 3), each product would be z_x^2, and the sum would be exactly n - 1. Dividing by n - 1 makes that perfect case come out as exactly 1.

What the formula explains.

No units. Every z-score is a pure number, so their products and r are too.
Symmetry. z_x z_y = z_y z_x, so swapping which variable is x gives the same r.
Range. A square is never negative, so

\begin{aligned} 0 \le \sum (z_x - z_y)^2 &= \sum z_x^2 - 2\sum z_x z_y + \sum z_y^2 \\ &= (n-1) - 2(n-1)r + (n-1) \\ &= 2(n-1)(1 - r). \end{aligned}

Since n - 1 > 0, this forces r \le 1. Equality needs every square to be 0, that is z_y = z_x for every point, which says \displaystyle y - \bar{y} = \frac{s_y}{s_x}(x - \bar{x}): every point on one line with positive slope. The same argument with z_x + z_y gives r \ge -1 (Exercise 14), with equality only for points on one falling line.

These are the facts that Properties of the correlation coefficient puts to work.

Worked examples

Example 1

Levan’s alpacas

Levan measures five alpacas (made-up data): shoulder height x in centimeters and fleece weight y in kilograms at shearing.

Alpaca	x (cm)	y (kg)
1	85	2.0
2	89	2.8
3	93	2.4
4	96	3.1
5	97	3.2

Find r from the z-scores.

Solution

The means are \displaystyle \bar{x} = \frac{460}{5} = 92 cm and \displaystyle \bar{y} = \frac{13.5}{5} = 2.7 kg. The deviations from the mean are -7, -3, 1, 4, 5 for height and -0.7, 0.1, -0.3, 0.4, 0.5 for weight. Their squares add to 100 and to 1.00, so

s_x = \sqrt{\tfrac{100}{4}} = 5 \text{ cm}, \qquad s_y = \sqrt{\tfrac{1.00}{4}} = 0.5 \text{ kg}.

Dividing each deviation by its standard deviation gives the z-scores, and the last column multiplies them.

Alpaca	z_x	z_y	z_x z_y
1	-1.4	-1.4	1.96
2	-0.6	0.2	-0.12
3	0.2	-0.6	-0.12
4	0.8	0.8	0.64
5	1.0	1.0	1.00

The products add to 3.36, and n - 1 = 4, so

r = \frac{3.36}{4} = 0.84 .

Figure 1 shows where this comes from. Alpacas 1, 4 and 5 sit in plus boxes, and alpaca 1, far below both means, gives more than half the total by itself. Alpacas 2 and 3 sit just across a center line in minus boxes, but each is close to a center line, so their products are small. A fairly strong positive straight-line relationship: taller alpacas in this group tend to carry heavier fleece.

Example 2

Anahit’s pens

Anahit keeps alpacas in five pens of equal size and measures the grass height y (cm) left after a week (made-up data). With x the number of alpacas in the pen, the pairs are (2, 12), (4, 12), (6, 10), (6, 8) and (7, 8). Predict the sign of r from the quadrants, then find r.

Solution

\displaystyle \bar{x} = \frac{25}{5} = 5 and \displaystyle \bar{y} = \frac{50}{5} = 10. The deviations are -3, -1, 1, 1, 2 for alpacas and 2, 2, 0, -2, -2 for grass, each with squares adding to 16, so \displaystyle s_x = s_y = \sqrt{16/4} = 2.

The two pens with fewer alpacas than average have taller grass than average (top-left box), and the pens with more alpacas have grass at or below average (bottom-right box or on the line). Every product will be negative or 0, so r < 0. Figure 2 draws the pens in z-units.

Figure 2Anahit’s pens (made-up data), drawn in z-units. Every dot is in a minus box or on an axis.

Pen	z_x	z_y	z_x z_y
1	-1.5	1	-1.5
2	-0.5	1	-0.5
3	0.5	0	0
4	0.5	-1	-0.5
5	1	-1	-1

The sum is -3.5, so \displaystyle r = \frac{-3.5}{4} = -0.875. A strong negative straight-line pattern: more alpacas in a pen go with shorter grass. Pen 3 has exactly average grass and adds nothing.

Example 3

A perfect line gives exactly 1

Aram buys feed bags at 8 dollars a bag plus a 12 dollar delivery fee. Orders of 1, 3, 5 and 7 bags cost 20, 36, 52 and 68 dollars. Show from the formula that r = 1.

Solution

\bar{x} = 4 bags and \bar{y} = 44 dollars. The bag deviations are -3, -1, 1, 3 and the cost deviations -24, -8, 8, 24, each exactly 8 times the bag deviation, because each extra bag adds 8 dollars. Multiplying every deviation by 8 multiplies the standard deviation by 8, so s_y = 8 s_x and

z_y = \frac{8(x - \bar{x})}{8 s_x} = z_x .

Each product is z_x^2. Here \displaystyle s_x^2 = \frac{9 + 1 + 1 + 9}{3} = \frac{20}{3}, so

\sum z_x z_y = \sum z_x^2 = \frac{20}{20/3} = 3 = n - 1,

and \displaystyle r = \frac{3}{3} = 1. The fee of 12 dollars and the price of 8 dollars both drop out.

Example 4

Rounded z-scores from technology

Anahit records age x (years) and fleece fiber diameter y (microns) for six alpacas (made-up data): (2, 19), (3, 22), (5, 21), (6, 24), (8, 27), (12, 26). A calculator gives \bar{x} = 6, s_x \approx 3.633, \bar{y} \approx 23.17, s_y \approx 3.061 and r \approx 0.85. Compute r from z-scores rounded to two decimal places and compare.

Solution

Each z-score uses the unrounded mean and standard deviation, then is rounded; for example \displaystyle z_x = \frac{2 - 6}{3.633} \approx -1.10 and \displaystyle z_y = \frac{19 - 23.17}{3.061} \approx -1.36.

Age	z_x	z_y	z_x z_y
2	-1.10	-1.36	1.50
3	-0.83	-0.38	0.32
5	-0.28	-0.71	0.20
6	0.00	0.27	0.00
8	0.55	1.25	0.69
12	1.65	0.93	1.53

The sum is 4.24 and n - 1 = 5, so \displaystyle r \approx \frac{4.24}{5} = 0.848 \approx 0.85, matching the calculator. Rounding the z-scores can move the third decimal of r, which is why technology is used for the final value. The six-year-old is exactly average in age and contributes 0 however unusual its fleece. Older alpacas in this group tend to have coarser fiber.

Example 5

Working backward to a missing product

Levan finds r = 0.6 for five alpacas. Four of the products z_x z_y are 1.2, 0.3, -0.2 and 0.5. Find the fifth. That alpaca has z_x = 1.5; find its z_y and say where it sits.

Solution

Multiplying the formula by n - 1 = 4, the products must add to 4 \times 0.6 = 2.4. The four known ones add to 1.8, so the fifth is 2.4 - 1.8 = 0.6. Then \displaystyle z_y = \frac{0.6}{1.5} = 0.4. Both z-scores are positive: the alpaca is above average in both variables, in the top-right box, a little above average in y and well above in x.

Boundaries

The formula needs at least two points and some spread in each variable. If every x is the same, s_x = 0, no z-score exists, and r is undefined.
x and y must be numerical and paired: two measurements on each individual.
Products of z-scores reward only a straight-line pattern. A cloud shaped like an arch puts large positive and large negative products on its two sides and can give r = 0 (Exercise 8) despite a perfect curve.
The formula here is the sample version, with n - 1 in the standard deviations and in the divisor. A population version uses N in both places; the two give the same r, because the factors cancel.
Hand computation with rounded z-scores is reliable to about two decimal places.

Where this goes next

The same z-score picture builds the least-squares line. The slope is \displaystyle b = r \frac{s_y}{s_x} (Slope from r and the spreads, and regression to the mean): in z-units the best line has slope exactly r. The The least squares criterion and its line explains why that line is best, and r squared reads r^2 as the fraction of the variation in y that the line accounts for. The no-units and symmetry facts become tools in Properties of the correlation coefficient.

Exercises

From z-scores to r

Five alpacas have z-scores for age z_x: -1, -1, 0, 1, 1 and for weight z_y: -1.4, -0.6, 1.0, 0.2, 0.8, in the same order. Find r.
Software gives rounded z-scores for six alpacas, grazing time z_x: -1.4, -0.8, -0.2, 0.3, 0.9, 1.2 and lying-down time z_y: 1.4, 0.6, 0.4, -0.9, -0.3, -1.2, in the same order. Find r to two decimal places.

From data to r

Aram’s five alpacas weigh 59, 61, 65, 66 and 69 kg and eat 0.8, 1.2, 1.9, 2.0 and 1.6 kg of hay a day, in that order. Find s_x, s_y and r.
Five alpacas aged 3, 5, 9, 10 and 13 years spat 4, 3, 4, 2 and 2 times during a vet visit. Find r and describe the relationship.
For the data of Exercise 3, find \displaystyle \frac{1}{n-1}\sum (x - \bar{x})(y - \bar{y}) with its units, then divide it by s_x s_y. What do you get, and what happens to each number if hay is recorded in grams?

Signs around the center

Five points have (z_x, z_y) equal to (1.2, -0.3), (-0.8, -1.1), (0, 2.0), (-1.5, 0.4) and (0.6, 0.9). Give the sign of each product and the box each point is in.
In Exercise 4, which alpaca’s product has the opposite sign from r, and which contributes nothing? Say why in words.
Hours after sunrise x = 1, 2, 3, 4, 5 and alpacas lying in the shade y = 1, 4, 5, 4, 1. Find r and describe what it misses.

How far r can go

A water trough holds 100, 70, 40 and 10 liters at 0, 2, 4 and 6 hours. Show from the formula that r = -1.
For six alpacas r = 0.5, and five of the products are 1.1, 0.4, -0.3, 0.2 and 0.6. Find the sixth. If that alpaca has z_x = -1.25, find its z_y.
A table for four points has products adding to 3.6. Without seeing it, explain why it holds a mistake.

Reasoning

Use the formula to explain why r has no units.
Use the formula to explain why swapping x and y leaves r unchanged.
Expand \sum (z_x + z_y)^2 \ge 0 to show r \ge -1, and say when r = -1.
Why does the formula divide by n - 1 rather than n? What would five points on a rising line give if it divided by n?
A sixth alpaca measuring exactly 92 cm and 2.7 kg joins Levan’s five. Show that r stays 0.84, and explain why a point at the center never changes r.

Answers

r = 0.75. Products 1.4, 0.6, 0, 0.2, 0.8 add to 3.0; \displaystyle \frac{3.0}{4} = 0.75.
r \approx -0.90. Products -1.96, -0.48, -0.08, -0.27, -0.27, -1.44 add to -4.50; \displaystyle \frac{-4.50}{5} = -0.90. Strong and negative: more grazing goes with less lying down.
s_x = 4 kg, s_y = 0.5 kg, r = 0.7875. Means 64 and 1.5; deviations -5, -3, 1, 2, 5 (squares add to 64) and -0.7, -0.3, 0.4, 0.5, 0.1 (squares add to 1.00). z-scores -1.25, -0.75, 0.25, 0.5, 1.25 and -1.4, -0.6, 0.8, 1.0, 0.2; products 1.75, 0.45, 0.2, 0.5, 0.25 add to 3.15, and \displaystyle \frac{3.15}{4} = 0.7875.
r = -0.6875. Means 8 and 3; s_x = 4, s_y = 1. z-scores -1.25, -0.75, 0.25, 0.5, 1.25 and 1, 0, 1, -1, -1; products -1.25, 0, 0.25, -0.5, -1.25 add to -2.75, and \displaystyle \frac{-2.75}{4} = -0.6875. A moderate negative relationship: older alpacas tended to spit less.
\displaystyle \frac{6.3}{4} = 1.575 kg times kg, and \displaystyle \frac{1.575}{4 \times 0.5} = 0.7875 = r. Deviation products 3.5, 0.9, 0.4, 1.0, 0.5 add to 6.3. In grams the first number becomes 1575 and s_y becomes 500, but the ratio is still 0.7875: dividing by s_x s_y removes the units.
Negative (bottom right); positive (bottom left); 0 (on the vertical center line); negative (top left); positive (top right).
The 9-year-old: z_x = 0.25, z_y = 1, product 0.25 > 0 while r < 0; a bit older than average yet spat more than average. The 5-year-old has z_y = 0 (exactly average spitting) and contributes 0.
r = 0. Deviations -2, -1, 0, 1, 2 and -2, 1, 2, 1, -2; products 4, -1, 0, 1, -4 add to 0. The pattern is a strong arch, but it is not a straight line, so r sees nothing.
Hour deviations -3, -1, 1, 3; liter deviations 45, 15, -15, -45, each -15 times. So s_y = 15 s_x and z_y = -z_x; the products add to -\sum z_x^2 = -3, and \displaystyle r = \frac{-3}{3} = -1.
0.5, and z_y = -0.4. The products must add to 5 \times 0.5 = 2.5; the five add to 2.0. \displaystyle z_y = \frac{0.5}{-1.25} = -0.4, bottom-left box.
It would give \displaystyle r = \frac{3.6}{3} = 1.2, but r \le 1, so the products can add to at most n - 1 = 3.
Each z-score divides a deviation by a standard deviation in the same units, so it is a pure number; products and averages of pure numbers are pure numbers.
Swapping turns each product z_x z_y into z_y z_x, the same number, so the sum and r do not change.
\sum (z_x + z_y)^2 = (n-1) + 2(n-1)r + (n-1), which is 2(n-1)(1 + r). This is \ge 0, so r \ge -1. Equality needs z_y = -z_x for every point: all points on one line with negative slope.
Because \sum z_x^2 = n - 1, dividing by n - 1 makes a perfect rising line give exactly 1. Dividing by n would give \displaystyle \frac{n-1}{n}, here \displaystyle \frac{4}{5} = 0.8, for a perfect line.
New \displaystyle s_x^2 = \frac{100}{5} = 20 and \displaystyle s_y^2 = \frac{1.00}{5} = 0.2. Each old product grows by the factor \displaystyle \frac{5}{4}, so the sum becomes \displaystyle 3.36 \times \frac{5}{4} = 4.2; the new alpaca adds 0, and \displaystyle \frac{4.2}{5} = 0.84. In general the point adds 0 to the sums of squares and of products, so both standard deviations shrink by the same factor and the change in the divisor cancels.