math.wiki
Probability › Random variables › Variance and covariance
Sign in

Variance and covariance

Two tollbooth lanes can take in the same average toll per car while one is far more predictable than the other, and the counts at two booths can rise together, move opposite ways, or ignore each other. What number measures spread around a mean, and what number measures how two random quantities move together?

Where you are 5 in Random variables

Why this exists

The school lessons Variance and standard deviation of a random variable and Combining random variables measured the spread of a discrete variable and added variances of independent ones. A time with a density needs its spread as an integral, and many counts are sums of terms that affect one another, such as hazardous loads among trucks picked without replacement. Expectation, linearity and indicators found the mean of such a count; its variance needs one more number for each pair of terms, the covariance, which measures how two variables move together.

Definition

Let X have a finite mean \mu_X. Its variance is

\operatorname{Var}(X) = E\big((X - \mu_X)^2\big),

a number in [0, \infty], computed by Result 1 of Expectation, linearity and indicators as \sum_x (x - \mu_X)^2 \, p_X(x) or \int (x - \mu_X)^2 f_X(x) \, dx. Its standard deviation is \displaystyle \sigma_X = \sqrt{\operatorname{Var}(X)}. A variable with no finite mean has no variance.

Let X and Y be random variables on the same sample space \Omega with finite variances. Their covariance is

\operatorname{Cov}(X, Y) = E\big((X - \mu_X)(Y - \mu_Y)\big),

which is finite by Result 3. They are uncorrelated if \operatorname{Cov}(X, Y) = 0. If also \sigma_X > 0 and \sigma_Y > 0, their correlation is \displaystyle \rho(X, Y) = \frac{\operatorname{Cov}(X, Y)}{\sigma_X \sigma_Y}.

Intuition

Variance is the mean squared distance from the balance point, the moment of inertia of a unit of mass about its center. A far value counts by the square of its distance, so a rare extreme value can carry most of the variance.

Put the point (\mu_X, \mu_Y) at the center of the plane. An outcome with both variables above their means, or both below, makes (X - \mu_X)(Y - \mu_Y) positive; one above and one below makes it negative. Covariance is the probability-weighted balance of these signs. It sees only that straight-line tendency: in Figure 2, Y is fixed by X, yet \operatorname{Cov}(X, Y) = 0.

Results

Result 1 (the shortcut, and zero variance). Let X have a finite mean. Its variance is finite exactly when E(X^2) is, and then

\begin{aligned}\operatorname{Var}(X) &= E(X^2) - \mu_X^2 \\ &= E\big(X(X - 1)\big) + \mu_X - \mu_X^2.\end{aligned}

\operatorname{Var}(X) = 0 exactly when P(X = \mu_X) = 1.

Result 2 (linear change). For constants a and b, \operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X) and \sigma_{aX + b} = |a| \, \sigma_X, the rule of Linear transformations of a random variable.

Result 3 (covariance). If X and Y have finite variances, then XY has a finite mean, \operatorname{Cov}(X, Y) is finite, and

\operatorname{Cov}(X, Y) = E(XY) - \mu_X \mu_Y.

Covariance is symmetric, \operatorname{Cov}(X, X) = \operatorname{Var}(X), \operatorname{Cov}(X, c) = 0 for a constant c, and it is linear in each slot: for constants a, b, c and Z with finite variance,

\begin{aligned}&\operatorname{Cov}(aX + bY + c, Z) \\ &\quad = a \operatorname{Cov}(X, Z) + b \operatorname{Cov}(Y, Z).\end{aligned}

Result 4 (variance of a sum). If X_1, \dots, X_n have finite variances, so does their sum, and

\begin{aligned}\operatorname{Var}\Big(\sum_{i=1}^{n} X_i\Big) &= \sum_{i=1}^{n} \operatorname{Var}(X_i) \\ &\quad + 2 \sum_{i < j} \operatorname{Cov}(X_i, X_j).\end{aligned}

In particular \operatorname{Var}(X \pm Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) \pm 2 \operatorname{Cov}(X, Y).

Result 5 (independence). If X and Y are independent with finite means, then XY has a finite mean and E(XY) = E(X) E(Y). So independent variables with finite variances are uncorrelated, and their variances add. If X_1, \dots, X_n are independent, each with variance \sigma^2, their average \displaystyle \bar{X} = \frac{1}{n}(X_1 + \dots + X_n) has \displaystyle \operatorname{Var}(\bar{X}) = \frac{\sigma^2}{n}.

Result 6 (correlation). -1 \le \rho(X, Y) \le 1. \rho(X, Y) = 1 exactly when Y = aX + b with probability 1 for some a > 0 and some b, and \rho(X, Y) = -1 exactly when this holds with a < 0. For data the same number is The correlation coefficient.

Result 7 (the families). A \operatorname{Binomial}(n, p) variable has variance np(1 - p), as in Mean and standard deviation of a binomial; a \operatorname{Geometric}(p) variable has variance \displaystyle \frac{1 - p}{p^2}, as in Mean and standard deviation of a geometric random variable; a \operatorname{Poisson}(\lambda) variable has variance \lambda, equal to its mean. \operatorname{Uniform}(a, b) has variance \displaystyle \frac{(b - a)^2}{12}, and \operatorname{Exponential}(\lambda) has variance \displaystyle \frac{1}{\lambda^2}. The means are in Result 6 of Discrete distributions: binomial, geometric, Poisson and Result 5 of Expectation, linearity and indicators.

Why it is true

Two facts about means are used throughout. Monotonicity: if U \ge V and both have finite means, then E(U) - E(V) = E(U - V) \ge 0 by linearity. Domination: if |U| \le V and V has a finite mean, so does U. For discrete variables this compares sums term by term, for a function of one variable with a density it compares integrals, and in general it is a measure-theoretic theorem used here without proof.

Result 1. Since (u + v)^2 \le 2u^2 + 2v^2, both (x - \mu_X)^2 \le 2x^2 + 2\mu_X^2 and x^2 \le 2(x - \mu_X)^2 + 2\mu_X^2, so domination shows that E(X^2) and \operatorname{Var}(X) are finite together. Taking means in (X - \mu_X)^2 = X^2 - 2\mu_X X + \mu_X^2 gives E(X^2) - \mu_X^2, and X^2 = X(X - 1) + X gives the second form.

If P(X = \mu_X) = 1, then (X - \mu_X)^2 = 0 with probability 1, so its mean is 0. Conversely, let \operatorname{Var}(X) = 0 and \varepsilon > 0. At every outcome (X - \mu_X)^2 \ge \varepsilon^2 \, \mathbf{1}_{\{|X - \mu_X| \ge \varepsilon\}}, so monotonicity and E(\mathbf{1}_A) = P(A) give 0 \ge \varepsilon^2 P(|X - \mu_X| \ge \varepsilon). The events \displaystyle \{|X - \mu_X| \ge \frac{1}{n}\} grow with n to \{X \ne \mu_X\}, so by continuity (Result 8 of Probability axioms and sample spaces) P(X \ne \mu_X) = 0.

Result 2. By linearity aX + b - (a\mu_X + b) = a(X - \mu_X), whose square has mean a^2 \operatorname{Var}(X).

Result 3. Expanding (|u| - |v|)^2 \ge 0 gives \displaystyle |uv| \le \frac12 (u^2 + v^2). With u = x - \mu_X, v = y - \mu_Y, domination makes the covariance finite, and with u = x, v = y it makes E(XY) finite. Taking means in (X - \mu_X)(Y - \mu_Y) = XY - \mu_Y X - \mu_X Y + \mu_X \mu_Y gives E(XY) - \mu_X \mu_Y. Symmetry, \operatorname{Cov}(X, X) = \operatorname{Var}(X) and \operatorname{Cov}(X, c) = 0 are read off the definition. For linearity, W = aX + bY + c has W - \mu_W = a(X - \mu_X) + b(Y - \mu_Y); multiply by Z - \mu_Z and take means.

Result 4. Expanding \sum_{i<j} (u_i - u_j)^2 \ge 0 gives \big(\sum_i u_i\big)^2 \le n \sum_i u_i^2, so with u_i = X_i - \mu_{X_i} the sum has a finite variance by domination. Linearity in both slots gives \operatorname{Cov}\big(\sum_i X_i, \sum_j X_j\big) = \sum_i \sum_j \operatorname{Cov}(X_i, X_j). The terms with i = j are the variances, and each pair i \ne j appears twice with the same value.

Result 5. Let X and Y be discrete. By Result 4 of Independence of events and variables, P(X = x, Y = y) = p_X(x) \, p_Y(y). Result 1 of Expectation, linearity and indicators, applied to the pair (X, Y) and g(x, y) = xy, needs

\begin{aligned}&\sum_{x, y} |xy| \, p_X(x) \, p_Y(y) \\ &\quad = \Big(\sum_x |x| \, p_X(x)\Big)\Big(\sum_y |y| \, p_Y(y)\Big) < \infty,\end{aligned}

and the same computation without absolute values, allowed for an absolutely convergent double series, gives E(XY) = E(X) E(Y). For variables with densities the product rule is proved in Joint distributions; for every independent pair it is a theorem of measure-theoretic probability, stated here without proof. Results 3 and 4 finish. For the average, the X_i are pairwise independent, so Results 2 and 4 give \displaystyle \operatorname{Var}(\bar{X}) = \frac{1}{n^2} \cdot n\sigma^2.

Result 6. Let U = X / \sigma_X and V = Y / \sigma_Y. By Results 2 and 3, \operatorname{Var}(U) = \operatorname{Var}(V) = 1 and \operatorname{Cov}(U, V) = \rho, so Result 4 gives

0 \le \operatorname{Var}(U \mp V) = 2 \mp 2\rho,

and -1 \le \rho \le 1. If \rho = 1, then \operatorname{Var}(V - U) = 0, and Result 1 makes V - U a constant k with probability 1, that is, \displaystyle Y = \frac{\sigma_Y}{\sigma_X} X + k \sigma_Y. If \rho = -1, the same applies to U + V. Conversely, if Y = aX + b with a \ne 0, then \operatorname{Cov}(X, Y) = a \operatorname{Var}(X) by Result 3 and \sigma_Y = |a| \sigma_X by Result 2, so \displaystyle \rho = \frac{a}{|a|}.

Result 7. Binomial: the count is \mathbf{1}_{A_1} + \dots + \mathbf{1}_{A_n} for success on n independent trials, and these indicators are independent (Result 6 of Independence of events and variables). Since \mathbf{1}_A^2 = \mathbf{1}_A, Result 1 gives \operatorname{Var}(\mathbf{1}_A) = P(A) - P(A)^2, and Result 5 gives np(1 - p).

Geometric, with q = 1 - p: differentiating the geometric series \displaystyle \sum_{k \ge 0} q^k = \frac{1}{1 - q} twice term by term, allowed for |q| < 1, gives \displaystyle \sum_{k \ge 2} k(k - 1) q^{k - 2} = \frac{2}{(1 - q)^3}. So

E\big(X(X - 1)\big) = \sum_{k \ge 2} k(k - 1) q^{k - 1} p = \frac{2q}{p^2},

and Result 1 gives \displaystyle \frac{2q}{p^2} + \frac{1}{p} - \frac{1}{p^2} = \frac{q}{p^2}, as 2q + p - 1 = q.

Poisson: \displaystyle E\big(X(X - 1)\big) = \lambda^2 e^{-\lambda} \sum_{k \ge 2} \frac{\lambda^{k - 2}}{(k - 2)!} = \lambda^2, so the variance is \lambda^2 + \lambda - \lambda^2 = \lambda.

Uniform: \displaystyle E(X^2) = \int_a^b \frac{x^2 \, dx}{b - a} = \frac{a^2 + ab + b^2}{3}, and subtracting \displaystyle \frac{(a + b)^2}{4} leaves \displaystyle \frac{(b - a)^2}{12}. Exponential: by parts, \displaystyle E(T^2) = \int_0^\infty 2t e^{-\lambda t} \, dt = \frac{2}{\lambda^2}, and \displaystyle \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2}.

Worked examples

Example 1

Axles and tolls

At Hannah’s tollbooth the number X of axles on a vehicle has P(X = 2) = 0.7, P(X = 3) = 0.1 and P(X = 5) = 0.2. The toll is \$1.50 per axle plus \$2. Find \operatorname{Var}(X) from the definition and by the shortcut, and the mean and standard deviation of the toll.

Solution

E(X) = 2(0.7) + 3(0.1) + 5(0.2) = 2.7. From the definition,

\begin{aligned}\operatorname{Var}(X) &= (-0.7)^2(0.7) + (0.3)^2(0.1) + (2.3)^2(0.2) \\ &= 0.343 + 0.009 + 1.058 = 1.41.\end{aligned}

By Result 1, E(X^2) = 4(0.7) + 9(0.1) + 25(0.2) = 8.7, and 8.7 - 2.7^2 = 1.41 again, so \displaystyle \sigma_X = \sqrt{1.41} \approx 1.187 axles. The five-axle trucks are one vehicle in five but supply 1.058 of the 1.41: far values weigh heavily.

The toll is C = 1.5X + 2 dollars, so E(C) = 1.5(2.7) + 2 = 6.05 dollars by linearity, and by Result 2, \operatorname{Var}(C) = 1.5^2(1.41) = 3.1725 squared dollars and \sigma_C = 1.5 \, \sigma_X \approx 1.78 dollars.

Example 2

Time at the weigh station

Trucks leaving Hannah’s booth pass a weigh station. The time S, in minutes, a truck spends there has density \displaystyle f_S(s) = \frac{s(6 - s)}{36} for 0 \le s \le 6, and 0 otherwise. Find \sigma_S and the probability that S lies within one standard deviation of its mean.

Solution

The density is symmetric about 3 and S is bounded, so E(S) = 3 (Exercise 15 of Expectation, linearity and indicators). By Result 1 there, with g(s) = s^2,

\begin{aligned}E(S^2) &= \int_0^6 \frac{s^3 (6 - s)}{36} \, ds \\ &= \frac{1}{36} \left(\frac{6^5}{4} - \frac{6^5}{5}\right) = \frac{54}{5},\end{aligned}

so \displaystyle \operatorname{Var}(S) = \frac{54}{5} - 9 = \frac{9}{5} and \displaystyle \sigma_S = \sqrt{1.8} \approx 1.342 minutes. With u = s - 3 the density is \displaystyle \frac{9 - u^2}{36}, so for 0 \le c \le 3,

P(|S - 3| \le c) = \int_{-c}^{c} \frac{9 - u^2}{36} \, du = \frac{c}{2} - \frac{c^3}{54}.

At \displaystyle c = \sqrt{1.8} this is about 0.626, the shaded band of Figure 1.

0 1 2 4 5 6 0.25 about 0.626
E(S) = 3
3 - \sigma_S
3 + \sigma_S
f_S(s)
s
Minutes at the weigh station
Figure 1The density \displaystyle f_S(s) = \frac{s(6 - s)}{36} of Example 2. The shaded band runs from 3 - \sigma_S to 3 + \sigma_S, with \displaystyle \sigma_S = \sqrt{1.8}, and carries probability about 0.626.
Example 3

Occupants and tolls

With carpool discounts at Hannah’s booth, the toll Y, in dollars, is 0, 2 or 4. For a random car, the number of occupants X and the toll have this joint pmf:

\begin{array}{c|ccc} & Y = 0 & Y = 2 & Y = 4 \\ \hline X = 1 & 0.05 & 0.15 & 0.30 \\ X = 2 & 0.10 & 0.15 & 0.05 \\ X = 3 & 0.10 & 0.05 & 0.05 \end{array}

Find \operatorname{Cov}(X, Y) and \rho(X, Y).

Solution

Row sums give p_X: 0.5, 0.3, 0.2. Column sums give p_Y: 0.25, 0.35, 0.40. So E(X) = 0.5 + 0.6 + 0.6 = 1.7 and E(Y) = 0.7 + 1.6 = 2.3. Result 1 of Expectation, linearity and indicators, applied to the pair, sums xy over the cells:

\begin{aligned}E(XY) &= 1(2 \cdot 0.15 + 4 \cdot 0.30) \\ &\quad + 2(2 \cdot 0.15 + 4 \cdot 0.05) \\ &\quad + 3(2 \cdot 0.05 + 4 \cdot 0.05) \\ &= 1.5 + 1.0 + 0.9 = 3.4.\end{aligned}

By Result 3, \operatorname{Cov}(X, Y) = 3.4 - 1.7(2.3) = -0.51: more occupants go with lower tolls. Next, E(X^2) = 0.5 + 1.2 + 1.8 = 3.5, so \operatorname{Var}(X) = 3.5 - 2.89 = 0.61, and E(Y^2) = 4(0.35) + 16(0.40) = 7.8, so \operatorname{Var}(Y) = 7.8 - 5.29 = 2.51. Then

\rho(X, Y) = \frac{-0.51}{\sqrt{0.61 \cdot 2.51}} \approx -0.412.
Example 4

Inspecting hazardous loads

Of the 40 trucks waiting at the booth one morning, 12 carry hazardous cargo. Jin picks 10 of them at random, without replacement, for a full inspection. Find the mean and variance of the number H of hazardous loads inspected, and compare with 10 independent picks.

Solution

Let A_i be the event that the i-th truck picked is hazardous. As in Exercise 14 of Expectation, linearity and indicators, \displaystyle P(A_i) = \frac{12}{40} = 0.3, so E(H) = 3. For i \ne j, the trucks in places i and j are equally likely to be any of the 40 \cdot 39 ordered pairs of different trucks, and 12 \cdot 11 pairs are both hazardous, so \displaystyle P(A_i \cap A_j) = \frac{132}{1560} = \frac{11}{130}. By Result 1, \operatorname{Var}(\mathbf{1}_{A_i}) = 0.3 - 0.09 = 0.21, and by Result 3,

\operatorname{Cov}(\mathbf{1}_{A_i}, \mathbf{1}_{A_j}) = \frac{11}{130} - 0.09 = -\frac{7}{1300},

negative because a hazardous pick leaves fewer for the others. Result 4, with 10 variances and \binom{10}{2} = 45 pairs, gives

\begin{aligned}\operatorname{Var}(H) &= 10(0.21) - 2 \cdot 45 \cdot \frac{7}{1300} \\ &= 2.1 - \frac{63}{130} = \frac{21}{13} \approx 1.615.\end{aligned}

With replacement the picks are independent and H \sim \operatorname{Binomial}(10, 0.3), with variance 2.1 by Result 7. Sampling without replacement multiplies it by \displaystyle \frac{30}{39} = \frac{40 - 10}{40 - 1}, a factor Exercise 18 proves in general.

The same route finishes the matching count of Expectation, linearity and indicators: its Exercise 19 gave E\big(N(N - 1)\big) = 1, and E(N) = 1, so Result 1 gives \operatorname{Var}(N) = 1 + 1 - 1 = 1 for every n \ge 2.

Example 5

Lanes and distance from the center

Malik’s toll plaza has five lanes, numbered -2, -1, 0, 1, 2 from left to right. A car uses lane X, with P(X = 0) = 0.3, P(X = 1) = P(X = -1) = 0.2 and P(X = 2) = P(X = -2) = 0.15, and Y = |X| is its distance in lanes from the center. Show that X and Y are uncorrelated but dependent, and that \operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) all the same.

Solution

The pmf of X is symmetric about 0, so E(X) = 0. In E(XY) = \sum_x x|x| \, p_X(x) the terms for x and -x cancel, so \operatorname{Cov}(X, Y) = 0 - 0 \cdot E(Y) = 0. Yet P(X = 0, Y = 0) = 0.3 while P(X = 0) \, P(Y = 0) = 0.09, so X and Y are dependent by Result 4 of Independence of events and variables.

−2 −1 0 1 2 1 2 0.15 0.2 0.3 0.2 0.15
E(Y) = 1
X
Y = |X|
Lane
Figure 2The values of (X, Y) in Example 5, each dot labeled with its probability. Y = |X| is fixed by X, yet the products (x - \mu_X)(y - \mu_Y) on the left arm cancel those on the right, so \operatorname{Cov}(X, Y) = 0.

By Result 4 the variances add, as the covariance is 0. Here \operatorname{Var}(X) = E(X^2) = 2(0.2) + 8(0.15) = 1.6, E(Y) = 0.4 + 0.6 = 1 and E(Y^2) = E(X^2), so \operatorname{Var}(Y) = 0.6 and \operatorname{Var}(X + Y) = 2.2.

Example 6

Two booths

In a five-minute span Malik’s booth serves X \sim \operatorname{Poisson}(6) cars and Jin’s serves Y \sim \operatorname{Poisson}(10), independently. Let T = X + Y. Find \operatorname{Var}(T), \operatorname{Cov}(X, T), \rho(X, T) and \operatorname{Cov}(X - Y, X + Y).

Solution

By Result 7, \operatorname{Var}(X) = 6 and \operatorname{Var}(Y) = 10; by Result 5, \operatorname{Cov}(X, Y) = 0, so \operatorname{Var}(T) = 16. Linearity in the second slot gives \operatorname{Cov}(X, T) = \operatorname{Var}(X) + \operatorname{Cov}(X, Y) = 6, so

\rho(X, T) = \frac{6}{\sqrt{6} \cdot 4} = \frac{\sqrt{6}}{4} \approx 0.612.

Expanding in both slots,

\begin{aligned}&\operatorname{Cov}(X - Y, X + Y) \\ &\quad = \operatorname{Var}(X) + \operatorname{Cov}(X, Y) \\ &\qquad - \operatorname{Cov}(Y, X) - \operatorname{Var}(Y) = -4.\end{aligned}

X - Y also has variance 16, so \rho(X - Y, X + Y) = -0.25, although X and Y are independent.

Example 7

The mean as the best forecast

Hannah forecasts the number X of cars in the next hour by one number c, and judges the forecast by the mean squared miss E\big((X - c)^2\big). Prove that if E(X^2) is finite, then for every constant c

E\big((X - c)^2\big) = \operatorname{Var}(X) + (\mu_X - c)^2,

so c = \mu_X, and only it, gives the smallest mean squared miss, \operatorname{Var}(X).

Solution

Write X - c = (X - \mu_X) + (\mu_X - c) and square:

\begin{aligned}(X - c)^2 &= (X - \mu_X)^2 + 2(\mu_X - c)(X - \mu_X) \\ &\quad + (\mu_X - c)^2.\end{aligned}

Each term has a finite mean by Result 1, and E(X - \mu_X) = 0, so linearity gives \operatorname{Var}(X) + 0 + (\mu_X - c)^2. The last term is at least 0, and is 0 only when c = \mu_X.

Boundaries

A variance can be infinite. If \displaystyle f_X(x) = \frac{2}{x^3} for x \ge 1, then \displaystyle E(X) = \int_1^\infty \frac{2}{x^2} \, dx = 2, but \displaystyle \int_1^b x^2 \cdot \frac{2}{x^3} \, dx = 2 \ln b \to \infty, so \operatorname{Var}(X) = \infty.

Uncorrelated is not independent. Independence gives covariance 0 (Result 5), not the converse (Example 5, Exercise 10). Only for two-valued variables do the two agree (Exercise 17).

Covariance carries units; correlation does not. With tolls in cents, the covariance of Example 3 becomes -51, while \rho is unchanged (Results 2 and 3).

Scope. Result 5 is proved here for discrete variables only; Joint distributions treats densities.

Where this goes next

Markov and Chebyshev inequalities turns a variance into a bound on P(|X - \mu_X| \ge k). The law of large numbers uses \displaystyle \operatorname{Var}(\bar{X}) = \frac{\sigma^2}{n} to show that averages settle at the mean. The bivariate normal takes \rho as a parameter, and Moment generating functions read E(X^2) off M_X''(0).

Exercises

Variances of one variable

  1. The number X of lanes open at Malik’s plaza at a random moment has P(X = 1) = 0.1, P(X = 2) = 0.4, P(X = 3) = 0.3 and P(X = 4) = 0.2. Find E(X), \operatorname{Var}(X) and \sigma_X. Each open lane costs \$35 per hour, plus \$50 per hour fixed; find the mean and standard deviation of the hourly cost.
  2. X has density f_X(x) = 4x^3 for 0 \le x \le 1, and 0 otherwise. Find \operatorname{Var}(X) and \sigma_X.
  3. The gap T, in seconds, between cars at a booth is \operatorname{Exponential}(0.5). Find E(T) and \operatorname{Var}(T), then P(|T - E(T)| \le \sigma_T) and P(T > E(T) + 2\sigma_T).
  4. A cash payment takes U \sim \operatorname{Uniform}(10, 40) seconds. Find \sigma_U and P(|U - E(U)| \le \sigma_U).
  5. Jin’s wait W, in minutes, at a booth is 0 with probability 0.6, and otherwise has density 0.08 e^{-0.2w} for w > 0. Find E(W), \operatorname{Var}(W) and \sigma_W.
  6. Each car at Jin’s booth, independently, has a transponder with probability 0.8. Find the standard deviation of the number of the next 50 cars with one; the mean, variance and standard deviation of the number G of cars up to and including the first one without; and the standard deviation of a \operatorname{Poisson}(4.5) count of oversized trucks.

Covariance, correlation and sums

  1. For a random vehicle at Hannah’s booth, X = 1 if it is a truck and 0 otherwise, and Y = 1 if it pays cash and 0 otherwise, with P(X = 0, Y = 0) = 0.4, P(X = 0, Y = 1) = 0.1, P(X = 1, Y = 0) = 0.2 and P(X = 1, Y = 1) = 0.3. Find \operatorname{Cov}(X, Y), \rho(X, Y) and \operatorname{Var}(X + Y). Are X and Y independent?
  2. \operatorname{Var}(X) = 9, \operatorname{Var}(Y) = 4 and \operatorname{Cov}(X, Y) = -2. Find \operatorname{Var}(X + Y), \operatorname{Var}(X - Y), \operatorname{Var}(2X - 3Y + 5) and \rho(X, Y).
  3. X \sim \operatorname{Uniform}(0, 1). Find \operatorname{Cov}(X, X^2) and \rho(X, X^2).
  4. X \sim \operatorname{Uniform}(-1, 1) and Y = X^2. Show that \operatorname{Cov}(X, Y) = 0 and that X and Y are dependent.
  5. Each of 10 cars picks one of 4 open lanes, uniformly and independently. Find the mean and variance of the number N of lanes that no car uses.
  6. Find the variance and standard deviation of the average of 12 independent payment times, each \operatorname{Uniform}(10, 40) seconds.
  7. Of 10 cars, X \sim \operatorname{Binomial}(10, 0.3) pay cash and Y = 10 - X do not. Find \operatorname{Cov}(X, Y) and \rho(X, Y).

Proofs

  1. Let a \le X \le b. Prove that \displaystyle \operatorname{Var}(X) \le \frac{(b - a)^2}{4}, and give a variable for which equality holds.
  2. Let X_1, \dots, X_n be independent, each with variance \sigma^2, with average \bar{X}. Prove that \operatorname{Cov}(\bar{X}, X_i - \bar{X}) = 0 for each i.
  3. Let \operatorname{Var}(X) = \operatorname{Var}(Y), finite. Prove that X + Y and X - Y are uncorrelated.
  4. Let X take only the values x_0 < x_1 and Y only the values y_0 < y_1. Prove that if X and Y are uncorrelated, they are independent.
  5. A bag holds M \ge 2 balls, K of them marked, and n \le M are drawn without replacement. Prove that the number of marked balls drawn has variance \displaystyle n \frac{K}{M}\left(1 - \frac{K}{M}\right) \frac{M - n}{M - 1}.
  6. Refute the claim that if \rho(X, Y) > 0 and \rho(Y, Z) > 0, then \rho(X, Z) > 0.
Answers
  1. E(X) = 2.6, E(X^2) = 7.6, \operatorname{Var}(X) = 7.6 - 6.76 = 0.84 and \sigma_X \approx 0.9165. By Result 2 the cost 35X + 50 has mean 35(2.6) + 50 = 141 dollars, variance 35^2(0.84) = 1029 and standard deviation 35 \sigma_X \approx 32.08 dollars.
  2. \displaystyle E(X) = \int_0^1 4x^4 \, dx = \frac45 and \displaystyle E(X^2) = \int_0^1 4x^5 \, dx = \frac23, so \displaystyle \operatorname{Var}(X) = \frac23 - \frac{16}{25} = \frac{2}{75} and \sigma_X \approx 0.1633.
  3. By Result 7, E(T) = 2, \operatorname{Var}(T) = 4 and \sigma_T = 2. Then P(T \le 4) = 1 - e^{-2} \approx 0.8647 and P(T > 6) = e^{-3} \approx 0.0498.
  4. \displaystyle \sigma_U = \frac{30}{\sqrt{12}} = 5\sqrt{3} \approx 8.660. The interval \displaystyle 25 \pm 5\sqrt{3} lies inside [10, 40], so the probability is \displaystyle \frac{10\sqrt{3}}{30} = \frac{1}{\sqrt{3}} \approx 0.5774.
  5. The mass at 0 adds nothing to either mean. \displaystyle E(W) = \int_0^\infty 0.08 \, w e^{-0.2w} \, dw = \frac{0.08}{0.04} = 2 and \displaystyle E(W^2) = 0.08 \cdot \frac{2}{0.008} = 20, so \operatorname{Var}(W) = 16 and \sigma_W = 4.
  6. The count is \operatorname{Binomial}(50, 0.8), with variance 50(0.8)(0.2) = 8 and standard deviation \approx 2.828. G \sim \operatorname{Geometric}(0.2) has mean 5, variance \displaystyle \frac{0.8}{0.04} = 20 and standard deviation \approx 4.472. The truck count has standard deviation \displaystyle \sqrt{4.5} \approx 2.121.
  7. E(X) = 0.5, E(Y) = 0.4 and E(XY) = 0.3, so \operatorname{Cov}(X, Y) = 0.3 - 0.2 = 0.1. With \operatorname{Var}(X) = 0.25 and \operatorname{Var}(Y) = 0.24, \displaystyle \rho(X, Y) = \frac{0.1}{\sqrt{0.06}} \approx 0.408 and \operatorname{Var}(X + Y) = 0.25 + 0.24 + 0.2 = 0.69. They are dependent, since independent variables have covariance 0.
  8. \operatorname{Var}(X + Y) = 9 + 4 - 4 = 9; \operatorname{Var}(X - Y) = 9 + 4 + 4 = 17; \operatorname{Var}(2X - 3Y + 5) is 4(9) + 9(4) + 2(2)(-3)(-2) = 96; \displaystyle \rho(X, Y) = \frac{-2}{3 \cdot 2} = -\frac13.
  9. \operatorname{Cov}(X, X^2) = E(X^3) - E(X)E(X^2), which is \displaystyle \frac14 - \frac16 = \frac{1}{12}. With \displaystyle \operatorname{Var}(X) = \frac{1}{12} and \displaystyle \operatorname{Var}(X^2) = \frac15 - \frac19 = \frac{4}{45}, \displaystyle \rho = \frac{\sqrt{15}}{4} \approx 0.968: not 1, as X^2 is not a linear function of X (Result 6).
  10. E(X) = 0 and \displaystyle E(X^3) = \int_{-1}^{1} \frac{x^3}{2} \, dx = 0, so \operatorname{Cov}(X, Y) = 0. But \displaystyle Y < \frac14 forces \displaystyle |X| < \frac12, so \displaystyle P(X > \frac12, Y < \frac14) = 0, while \displaystyle P(X > \frac12) \, P(Y < \frac14) = \frac14 \cdot \frac12 = \frac18.
  11. Let B_i be the event that lane i is unused: \displaystyle P(B_i) = \left(\frac34\right)^{10} = a and, for i \ne j, \displaystyle P(B_i \cap B_j) = \left(\frac12\right)^{10} = b. Then E(N) = 4a \approx 0.2253, and Results 1, 3 and 4 give \operatorname{Var}(N) = 4a(1 - a) + 12(b - a^2) \approx 0.1862, counting 12 ordered pairs.
  12. By Results 5 and 7, the variance is \displaystyle \frac{75}{12} = 6.25 square seconds, and the standard deviation is 2.5 seconds.
  13. By Result 3, \operatorname{Cov}(X, 10 - X) = -\operatorname{Var}(X), which is -10(0.3)(0.7) = -2.1, and \operatorname{Var}(Y) = \operatorname{Var}(X) by Result 2, so \rho(X, Y) = -1, as Result 6 requires.
  14. X is bounded, so E(X^2) is finite. With \displaystyle c = \frac{a + b}{2}, \displaystyle (X - c)^2 \le \frac{(b - a)^2}{4} always, so Example 7 and monotonicity give \displaystyle \operatorname{Var}(X) \le E\big((X - c)^2\big) \le \frac{(b - a)^2}{4}. Equality holds when X is a or b with probability \displaystyle \frac12 each, as then \mu_X = c.
  15. By linearity in the second slot and Result 5, \displaystyle \operatorname{Cov}(\bar{X}, X_i) = \frac1n \sum_j \operatorname{Cov}(X_j, X_i) = \frac{\sigma^2}{n}, since only j = i contributes. Also \displaystyle \operatorname{Cov}(\bar{X}, \bar{X}) = \operatorname{Var}(\bar{X}) = \frac{\sigma^2}{n}, so \operatorname{Cov}(\bar{X}, X_i - \bar{X}) = 0.
  16. Expanding in both slots, \operatorname{Cov}(X + Y, X - Y) is \operatorname{Var}(X) - \operatorname{Cov}(X, Y) + \operatorname{Cov}(Y, X) - \operatorname{Var}(Y). The middle terms cancel by symmetry, the outer ones are equal.
  17. Let A = \{X = x_1\} and B = \{Y = y_1\}, so X = x_0 + (x_1 - x_0)\mathbf{1}_A and Y = y_0 + (y_1 - y_0)\mathbf{1}_B. By Result 3 and \mathbf{1}_A \mathbf{1}_B = \mathbf{1}_{A \cap B}, \operatorname{Cov}(X, Y) is (x_1 - x_0)(y_1 - y_0) times P(A \cap B) - P(A)P(B). The first factor is not 0, so uncorrelated means A and B are independent. Then \mathbf{1}_A and \mathbf{1}_B are independent (Result 6 of Independence of events and variables), and so are X and Y, functions of them (Result 5 there).
  18. Let A_i be the event that draw i is marked and \displaystyle p = \frac{K}{M}, so P(A_i) = p (Exercise 14 of Expectation, linearity and indicators). For i \ne j, the balls in places i and j are equally likely to be any of the M(M - 1) ordered pairs of different balls, so \displaystyle P(A_i \cap A_j) = \frac{K(K - 1)}{M(M - 1)}, and subtracting p^2 leaves the covariance \displaystyle \frac{K(K - M)}{M^2 (M - 1)} = -\frac{p(1 - p)}{M - 1}. Result 4, with n(n - 1) ordered pairs, gives \displaystyle np(1 - p) - n(n - 1)\frac{p(1 - p)}{M - 1}, which is \displaystyle np(1 - p) \frac{M - n}{M - 1}.
  19. Let X and Z be independent, each \pm 1 with probability \displaystyle \frac12, so each has variance 1, and let Y = X + Z. By Result 5 (they are discrete), \operatorname{Cov}(X, Z) = 0, so Results 3 and 4 give \operatorname{Cov}(X, Y) = \operatorname{Cov}(Y, Z) = 1 and \operatorname{Var}(Y) = 2. So \displaystyle \rho(X, Y) = \rho(Y, Z) = \frac{1}{\sqrt{2}} > 0, while \rho(X, Z) = 0.
math.wiki · CC-BY-SA-4.0