Conditional expectation

Once one quantity is known, such as how many presses ran today, the mean of another, such as how many barrels were filled, has to be recomputed. What is that updated mean, why is it itself random, and how do the updated means average back to the plain one?

Where you are 5 in Several random variables

Why this exists

At Naomi’s cidery the number X of barrels filled in a day depends on the number Y of presses running. Once Y is known, the mean of X should be taken over the conditional distribution of Joint distributions. That updated mean is a number for each value of Y, so before Y is seen it is itself a random variable.

Seen that way it becomes a tool. Many models are built in stages: a random rate, then a count at that rate; a random number of customers, then each one’s purchase. The mean and variance of the result come from the stages, case by case, without first finding its distribution.

Definition

In plain words, the conditional expectation of X given Y is the mean of X recomputed once Y is known. Throughout, X has a finite mean, and functions are regular enough for Result 6 of Joint distributions.

Given an event. For an event B with P(B) > 0,

E(X \mid B) = \frac{E(X \mathbf{1}_B)}{P(B)}.

For discrete X this is \sum_x x \, P(X = x \mid B), the mean of the conditional pmf (Finding a conditional probability), since X \mathbf{1}_B is x on \{X = x\} \cap B and 0 off B.

Given a value. Let Y be a second random variable.

  • Y discrete, p_Y(y) > 0: E(X \mid Y = y) = E(X \mid \{Y = y\}).
  • (X, Y) with a joint density, f_Y(y) > 0: E(X \mid Y = y) = \int x \, f_{X \mid Y}(x \mid y) \, dx, with the conditional density of Result 5 of Joint distributions.
  • A mixed pair as in Result 5(d) there, Y with a density and discrete X with model pmf p_{X \mid Y}: E(X \mid Y = y) = \sum_x x \, p_{X \mid Y}(x \mid y).

The sum or integral must converge absolutely. In the last two cases g(X, Y) in place of X puts g(x, y) in place of x, and P(A \mid Y = y) means E(\mathbf{1}_A \mid Y = y).

Given the variable. Let m(y) = E(X \mid Y = y) where it is defined, and m(y) = 0 elsewhere. The conditional expectation of X given Y is the random variable

E(X \mid Y) = m(Y).

When E(X^2) < \infty, the conditional variance is v(y) = E((X - m(y))^2 \mid Y = y), and \operatorname{Var}(X \mid Y) = v(Y). Expanding the square, by linearity of sums, integrals and E(\cdot \, \mathbf{1}_B),

\operatorname{Var}(X \mid Y = y) = E(X^2 \mid Y = y) - m(y)^2.

For discrete Y, m(y) exists wherever p_Y(y) > 0; with a density it can fail only on a set Y avoids with probability 1 (measure theory, used without proof), so no result depends on those values.

Intuition

For discrete Y, the events \{Y = y\} split \Omega into blocks, and E(X \mid Y) replaces X on each block by its average there. Averaging the block averages, weighted by the blocks’ probabilities, gives back the overall average: the tower rule, Result 1.

With a joint density, slice the support at height y. The conditional density is the slice rescaled to area 1, and m(y) is its balance point. As y moves, the balance points trace the regression curve of X on Y. Vertical slices give the regression curve of Y on X, in general a different curve (Figure 1).

0.5 0.5 1 1 0
x = \tfrac59 y
y = E(Y \mid X = x)
x
y
Figure 1The support of the joint density 2(x + y) of Example 2. The straight line x = \tfrac59 y joins the balance points of the horizontal slices, so it is the graph of E(X \mid Y = y); the dot marks the balance point \tfrac49 of the dashed slice at y = 0.8. The curve is y = E(Y \mid X = x), which is not a line.

Results

Result 1 (tower rule). If E(|X|) < \infty, then E(E(X \mid Y)) = E(X). For Y discrete or with a density,

\begin{aligned}E(X) &= \textstyle\sum_y E(X \mid Y = y) \, p_Y(y), \\ E(X) &= \int_{-\infty}^{\infty} E(X \mid Y = y) \, f_Y(y) \, dy.\end{aligned}

More generally, if W is a random variable (a function g(X, Y) when Y is not discrete) and E(|h(Y) W|) < \infty, then E(h(Y) W) = E(h(Y) \, E(W \mid Y)). With W = \mathbf{1}_A and h = 1 this is the The law of total probability, P(A) = E(P(A \mid Y)).

Result 2 (rules). Let W_1, W_2 have finite means (functions of (X, Y) when Y is not discrete), a, b constants and h a function. Wherever the conditional means exist:

(a) E(a W_1 + b W_2 \mid Y) = a E(W_1 \mid Y) + b E(W_2 \mid Y);

(b) if h(Y) W_1 has a finite mean, E(h(Y) W_1 \mid Y) = h(Y) \, E(W_1 \mid Y), and E(h(Y) \mid Y) = h(Y);

(c) if X and Y are independent, with Y discrete or a joint density, then E(X \mid Y) = E(X).

Result 3 (law of total variance). If E(X^2) < \infty, then E(X \mid Y) has finite variance and

\operatorname{Var}(X) = E(\operatorname{Var}(X \mid Y)) + \operatorname{Var}(E(X \mid Y)).

The spread of X is the average spread within the cases plus the spread of the case means.

Result 4 (best predictor). Let E(X^2) < \infty and m(Y) = E(X \mid Y). For every h with E(h(Y)^2) < \infty, E((X - m(Y)) h(Y)) = 0 and

\begin{aligned}&E((X - h(Y))^2) \\ &\quad = E((X - m(Y))^2) + E((m(Y) - h(Y))^2).\end{aligned}

So of all predictors of X computed from Y, E(X \mid Y) has the least mean squared error, E(\operatorname{Var}(X \mid Y)), and any h(Y) doing as well equals m(Y) with probability 1.

Result 5 (random sums). Let N, X_1, X_2, \dots be independent, N with values in \{0, 1, 2, \dots\} and finite variance, each X_i with mean \mu and variance \sigma^2. Let S = X_1 + \dots + X_N, with S = 0 when N = 0. Then E(S \mid N) = N\mu, \operatorname{Var}(S \mid N) = N\sigma^2, and

\begin{aligned}E(S) &= \mu \, E(N), \\ \operatorname{Var}(S) &= \sigma^2 E(N) + \mu^2 \operatorname{Var}(N).\end{aligned}
Why it is true

Result 1. Let m_W(y) = E(W \mid Y = y). We prove E(h(Y) W) = E(h(Y) m_W(Y)); h = 1 is the tower rule.

Discrete Y. On \{Y = y\}, h(Y) = h(y), so h(y) m_W(y) p_Y(y) = E(h(Y) W \mathbf{1}_{\{Y = y\}}). The events \{Y = y\} with p_Y(y) > 0 cover \Omega up to probability 0, so for finitely many values linearity (Expectation, linearity and indicators, Result 2) gives

\textstyle\sum_y E(h(Y) W \mathbf{1}_{\{Y = y\}}) = E(h(Y) W),

and by Result 1 there the left side is E(h(Y) m_W(Y)); the same sum with absolute values, at most E(|h(Y) W|), gives convergence. For countably many values, moving the sum inside E is the dominated convergence theorem (partial sums are bounded by |h(Y) W|), and for W \ge 0 it holds with both sides possibly infinite (monotone convergence); both are stated without proof (measure theory).

Joint density. As in the proof of Result 5(c) of Joint distributions, take f = 0 on each line y = t with f_Y(t) = 0. Then m_W(y) f_Y(y) = \int g(x, y) f(x, y) \, dx for all y outside a set Y avoids (Definition). Result 6 there, Fubini and Result 3 there give

\begin{aligned}E(h(Y) m_W(Y)) &= \int h(y) \, m_W(y) f_Y(y) \, dy \\ &= \int h(y) \int g f \, dx \, dy,\end{aligned}

which is E(h(Y) g(X, Y)) by Result 6 again. Fubini applies as \iint |h g| f \, dA = E(|h(Y) W|) < \infty, and |m_W| f_Y \le \int |g| f \, dx makes E(|h(Y) m_W(Y)|) finite.

Mixed pair. The same computation runs with a sum over x, using E(g(X, Y)) = \int \sum_x g(x, y) \, p_{X \mid Y}(x \mid y) f_Y(y) \, dy, the mixed form of Result 6 of Joint distributions, used here without proof (measure theory).

Result 2. (a) Sums, integrals and W \mapsto E(W \mathbf{1}_B) are linear. (b) For discrete Y, h(Y) W_1 \mathbf{1}_{\{Y = y\}} = h(y) W_1 \mathbf{1}_{\{Y = y\}}; otherwise h(y) factors out of the sum or integral over x. With W_1 = 1, the conditional pmf or density has total 1 (Joint distributions, Result 5(a)). (c) For discrete Y, X and \mathbf{1}_{\{Y = y\}} are independent, so Result 5 of Variance and covariance gives E(X \mathbf{1}_{\{Y = y\}}) = E(X) p_Y(y). With a density, Result 4(a) of Joint distributions gives f_{X \mid Y}(x \mid y) = f_X(x).

Result 3. As v \ge 0, m(y)^2 \le E(X^2 \mid Y = y), and Result 1 for X^2 gives E(m(Y)^2) \le E(X^2). By Result 1 again, and E(m(Y)) = E(X),

\begin{aligned}E(v(Y)) &= E(X^2) - E(m(Y)^2), \\ \operatorname{Var}(m(Y)) &= E(m(Y)^2) - E(X)^2.\end{aligned}

Adding gives E(X^2) - E(X)^2 = \operatorname{Var}(X).

Result 4. Let k = m - h. By Result 3 and (u - w)^2 \le 2u^2 + 2w^2, E(k(Y)^2) is finite, and \displaystyle |uw| \le \frac12 (u^2 + w^2) makes E(|k(Y) X|) and E(|h(Y) X|) finite. Result 1 gives E(h(Y) X) = E(h(Y) m(Y)), the orthogonality, and likewise for k. Writing X - h(Y) = (X - m(Y)) + k(Y) and expanding, the cross term vanishes, giving the identity; Result 1 for (X - m(Y))^2 gives E((X - m(Y))^2) = E(v(Y)). If h(Y) does as well, E(k(Y)^2) = 0, and the lemma in the proof of Result 5 of Markov and Chebyshev inequalities gives k(Y) = 0 with probability 1.

Result 5. Fix n with p_N(n) > 0 and let W = X_1 + \dots + X_n, so S = W on \{N = n\}. W is a function of X_1, \dots, X_n, so it is independent of \mathbf{1}_{\{N = n\}} (Independence of events and variables), and Result 5 of Variance and covariance gives E(S \mid N = n) = E(W) = n\mu and E(S^2 \mid N = n) = E(W^2). By Result 4 there, \operatorname{Var}(W) = n\sigma^2. Result 1 for S^2 \ge 0 gives E(S^2) = \sigma^2 E(N) + \mu^2 E(N^2) < \infty. Then Results 1 and 3 give E(S) = E(N\mu) and \operatorname{Var}(S) = E(N\sigma^2) + \operatorname{Var}(N\mu).

Worked examples

Example 1

Presses and barrels

Naomi’s cidery runs Y presses on a given day, one or two, and fills X barrels. The joint pmf is

x = 2x = 3x = 4x = 5
y = 1\tfrac{3}{20}\tfrac{4}{20}\tfrac{1}{20}0
y = 20\tfrac{2}{20}\tfrac{5}{20}\tfrac{5}{20}

Find E(X \mid Y = y), write E(X \mid Y) as a formula in Y, and check the tower rule and the law of total variance.

Solution

The rows give p_Y(1) = \tfrac25 and p_Y(2) = \tfrac35. Dividing each row by its total,

\begin{aligned}E(X \mid Y = 1) &= \tfrac{2 \cdot 3 + 3 \cdot 4 + 4 \cdot 1}{8} = \tfrac{11}{4}, \\ E(X \mid Y = 2) &= \tfrac{3 \cdot 2 + 4 \cdot 5 + 5 \cdot 5}{12} = \tfrac{17}{4}.\end{aligned}

So E(X \mid Y) is \tfrac{11}{4} with probability \tfrac25 and \tfrac{17}{4} with probability \tfrac35: as a formula, E(X \mid Y) = \tfrac54 + \tfrac32 Y. By Result 1, E(X) = \tfrac{11}{4} \cdot \tfrac25 + \tfrac{17}{4} \cdot \tfrac35 = \tfrac{73}{20}, as the column totals \tfrac{3}{20}, \tfrac{6}{20}, \tfrac{6}{20}, \tfrac{5}{20} also give.

Next, E(X^2 \mid Y = 1) = 8 and E(X^2 \mid Y = 2) = \tfrac{223}{12}, so the conditional variances are 8 - \tfrac{121}{16} = \tfrac{7}{16} and \tfrac{223}{12} - \tfrac{289}{16} = \tfrac{25}{48}, with mean \tfrac{7}{16} \cdot \tfrac25 + \tfrac{25}{48} \cdot \tfrac35 = \tfrac{39}{80}. As Y - 1 is an indicator, \operatorname{Var}(Y) = \tfrac{6}{25}, so \operatorname{Var}(E(X \mid Y)) = \tfrac94 \cdot \tfrac{6}{25} = \tfrac{27}{50} (Variance and covariance, Result 2). The sum is \tfrac{411}{400}, and directly \operatorname{Var}(X) = \tfrac{287}{20} - \tfrac{5329}{400} = \tfrac{411}{400}.

Example 2

Starting and stopping the press

The press starts at time X and stops at time Y, as fractions of the working day, with joint density f(x, y) = 2(x + y) for 0 < x < y < 1 and 0 elsewhere. Find E(X \mid Y = y) and E(Y \mid X = x), then E(X) and the mean running time E(Y - X) by the tower rule.

Solution

By Result 3 of Joint distributions, f_Y(y) = \int_0^y 2(x + y) \, dx = 3y^2 and f_X(x) = (1 - x)(1 + 3x), both on (0, 1). Given Y = y, X has density \displaystyle \frac{2(x + y)}{3y^2} on (0, y), so

\begin{aligned}E(X \mid Y = y) &= \frac{2}{3y^2} \int_0^y (x^2 + xy) \, dx \\ &= \frac{2}{3y^2} \cdot \frac{5y^3}{6} = \frac{5y}{9}.\end{aligned}

For 0 < x < 1, \int_x^1 2y(x + y) \, dy = \tfrac13 (1 - x)(5x^2 + 5x + 2), and dividing by f_X(x),

E(Y \mid X = x) = \frac{5x^2 + 5x + 2}{3(1 + 3x)}.

The first is linear and the second is not (Figure 1). As E(Y) = \int_0^1 3y^3 \, dy = \tfrac34, Result 1 gives E(X) = \tfrac59 \cdot \tfrac34 = \tfrac{5}{12}. By Result 2, E(Y - X \mid Y) = \tfrac49 Y, so E(Y - X) = \tfrac49 \cdot \tfrac34 = \tfrac13.

Example 3

How well the stop predicts the start

For the press of Example 2, find the mean squared error of predicting X by E(X \mid Y), and by the constant E(X).

Solution

From the conditional density, \displaystyle E(X^2 \mid Y = y) = \frac{7y^2}{18}, so

\operatorname{Var}(X \mid Y = y) = \frac{7y^2}{18} - \frac{25y^2}{81} = \frac{13y^2}{162}.

By Result 4, with E(Y^2) = \tfrac35, the error of \tfrac59 Y is \tfrac{13}{162} \cdot \tfrac35 = \tfrac{13}{270} \approx 0.0481. The best constant is E(X) (Example 7 of Variance and covariance), with error \operatorname{Var}(X) = \tfrac{7}{30} - \tfrac{25}{144} = \tfrac{43}{720} \approx 0.0597, as E(X^2) = \tfrac{7}{30}. Result 3 ties them: \operatorname{Var}(\tfrac59 Y) = \tfrac{25}{81} \cdot \tfrac{3}{80} = \tfrac{5}{432}, and \tfrac{13}{270} + \tfrac{5}{432} = \tfrac{43}{720}. Knowing Y removes the fraction \tfrac{25}{129} \approx 0.19 of the error.

Example 4

Caps from a varying box

Carlos seals bottles in cases of 24. Within a box of caps each cap seals with probability P, but P varies from box to box, P \sim \operatorname{Uniform}(0.8, 1); given P = p, the caps seal independently, so the number N sealed in a case is \operatorname{Binomial}(24, p). Find E(N), \operatorname{Var}(N) and P(N = 24), and compare with a fixed rate 0.9.

Solution

By Result 7 of Variance and covariance, E(N \mid P) = 24P and \operatorname{Var}(N \mid P) = 24P(1 - P). Here E(P) = \tfrac{9}{10} and E(P^2) = 5 \int_{0.8}^1 p^2 \, dp = \tfrac{61}{75}, so \operatorname{Var}(P) = \tfrac{1}{300}. By Results 1 and 3,

\begin{aligned}E(N) &= 24 \cdot \tfrac{9}{10} = 21.6, \\ E(\operatorname{Var}(N \mid P)) &= 24 \left( \tfrac{9}{10} - \tfrac{61}{75} \right) = \tfrac{52}{25}, \\ \operatorname{Var}(E(N \mid P)) &= 576 \cdot \tfrac{1}{300} = \tfrac{48}{25},\end{aligned}

so \operatorname{Var}(N) = 4. At the fixed rate, \operatorname{Var}(N) = 24 \cdot 0.9 \cdot 0.1 = 2.16. As P(N = 24 \mid P = p) = p^{24}, the tower rule for an indicator gives

\begin{aligned}P(N = 24) &= 5 \int_{0.8}^1 p^{24} \, dp \\ &= \frac{1 - 0.8^{25}}{5} \approx 0.1992,\end{aligned}

against 0.9^{24} \approx 0.0798 at the fixed rate (Figure 2).

14 16 18 20 22 24 0.1 0.2
P \sim \operatorname{Uniform}(0.8, 1)
p = 0.9
Sealed bottles in a case
Figure 2The number N of sealed bottles in Example 4. Shaded, the rate P varies from box to box, uniform on (0.8, 1); dashed, every cap seals with probability 0.9. Both have mean 21.6, but the varying rate raises the variance from 2.16 to 4 and gives a full case probability 0.1992 rather than 0.0798.
Example 5

Growers at the gate

The number N of growers who bring apples to the press on a Saturday is \operatorname{Poisson}(60). Grower i brings X_i bushels, with mean 18 and standard deviation 10, and N, X_1, X_2, \dots are independent. Find the mean and standard deviation of the total S, and bound P(|S - 1080| \ge 400).

Solution

N has mean and variance 60 (Variance and covariance, Result 7). By Result 5, E(S) = 18 \cdot 60 = 1080 bushels and

\operatorname{Var}(S) = 100 \cdot 60 + 324 \cdot 60 = 25{,}440,

so \displaystyle \sigma_S = \sqrt{25{,}440} \approx 159.50 bushels. Chebyshev’s inequality (Markov and Chebyshev inequalities, Result 3) gives

P(|S - 1080| \ge 400) \le \frac{25{,}440}{400^2} = 0.159.
Example 6

A constant conditional mean

Let E(X^2) and E(Y^2) be finite, and suppose E(X \mid Y) = c with probability 1 for a constant c. Prove that c = E(X) and \operatorname{Cov}(X, Y) = 0.

Solution

By Result 1, E(X) = E(E(X \mid Y)) = E(c) = c. By Result 3 of Variance and covariance, XY has a finite mean, so Result 1 with h(y) = y applies:

E(XY) = E(Y \, E(X \mid Y)) = E(cY) = E(X) E(Y).

Hence \operatorname{Cov}(X, Y) = 0. The converse fails (Exercise 16).

Example 7

Two good barrels in a row

Carlos tastes barrels one at a time, each passing independently with probability \tfrac34, and stops as soon as two consecutive barrels pass. Find the mean number T of barrels tasted.

Solution

E(T) is finite: T is at most twice the number of disjoint pairs of consecutive barrels up to the first pair that both pass, which is \operatorname{Geometric}(\tfrac{9}{16}). Let m = E(T).

The run starts in one of three ways: the first barrel fails (probability \tfrac14); it passes and the second fails (\tfrac{3}{16}); both pass (\tfrac{9}{16}). After a failure the later barrels form a fresh run, independent of the earlier ones, so in the first case T = 1 + T' with T' distributed as T, and Result 5 of Variance and covariance gives E(T \mid \text{first fails}) = 1 + m. The second case gives 2 + m, the third exactly 2. By Result 1,

m = \tfrac14 (1 + m) + \tfrac{3}{16} (2 + m) + \tfrac{9}{16} \cdot 2.

So \tfrac{9}{16} m = \tfrac74, and m = \tfrac{28}{9} \approx 3.11 barrels.

Boundaries

The tower rule needs a finite mean. Let Y \sim \operatorname{Exponential}(1) and, given Y = y, let X = \pm Z with a fair random sign and Z \sim \operatorname{Exponential}(y). Then E(X \mid Y) = 0, but \displaystyle E(|X| \mid Y) = \frac1Y and \displaystyle \int_0^\infty \frac{e^{-y}}{y} \, dy diverges, so E(X) does not exist.

A conditional mean belongs to the variable. Changing the density on one line changes m(y) there but not m(Y) with probability 1, and one event written through different variables gives different answers (the Borel paradox in Joint distributions).

Best means least squared error. For the mean absolute error the best predictor is a conditional median instead. The best straight line in Y equals E(X \mid Y) only when that is linear, as E(Y \mid X) in Example 2 is not.

Random sums need N independent of the summands. If the X_i are independent fair 0 or 1 variables and N = X_1, then S = X_1, so E(S) = \tfrac12, while \mu E(N) = \tfrac14.

Where this goes next

Conditioning on one summand gives the distribution of a sum: for independent X and Y with densities, P(X + Y \le s) = E(F_X(s - Y)), the convolution of Sums of independent variables. The tower rule applied to e^{tS} gives a random sum’s generating function. For the The bivariate normal, E(Y \mid X) is linear, so the best predictor is the Least squares regression line. The first-step argument of Example 7 finds hitting times in Markov chains.

Exercises

Conditional means

  1. A pump’s lifetime X, in years, is \operatorname{Exponential}(\tfrac13). Find E(X \mid X > 2) and E(X \mid X \le 2).
  2. X bruised apples are found in a sample from a crate, and Y = 1 if the crate came from the hillside orchard, Y = 0 if from the valley. The joint pmf is p(0, 0) = p(1, 0) = \tfrac16, p(2, 0) = 0, p(0, 1) = \tfrac{1}{12}, p(1, 1) = \tfrac14, p(2, 1) = \tfrac13. Find E(X \mid Y = y) for both y, E(X \mid Y) as a formula in Y, E(X), and E(Y \mid X = x) for x = 0, 1, 2.
  3. Naomi rolls a fair die twice. Let X be the first roll and M the larger of the two. Find E(M \mid X = x) and then E(M) by the tower rule.
  4. (X, Y) has joint density 6(1 - y) for 0 < x < y < 1, and 0 elsewhere. Find E(X \mid Y = y), \operatorname{Var}(X \mid Y = y), E(Y \mid X = x) and E(X).
  5. A bottling line runs for Y \sim \operatorname{Exponential}(2) hours before it jams, and given Y = y, the number X of cracked bottles is \operatorname{Poisson}(3y). Find E(X), \operatorname{Var}(X) and P(X = 0).

Total variance and random sums

  1. A batch ferments for Y \sim \operatorname{Exponential}(\tfrac12) days, and given Y = y, a sample is drawn at a time X \sim \operatorname{Uniform}(0, y). Find E(X) and \operatorname{Var}(X).
  2. Each of 50 apples in a bin is bruised with probability P, independently given P, where P \sim \operatorname{Uniform}(0, 1) varies by bin. Find the mean and variance of the number N bruised, and P(N = 50).
  3. A starter culture takes on each attempt with probability P, independently given P, where P \sim \operatorname{Uniform}(\tfrac12, 1) varies by culture. Let N be the number of attempts up to the first success. Find E(N), P(N = 1) and P(N = 2).
  4. The number of crates delivered is N \sim \operatorname{Binomial}(10, 0.8), and the crates weigh X_1, X_2, \dots pounds, with mean 40 and standard deviation 3, all independent. Find the mean and standard deviation of the total weight.
  5. Naomi presses batches until one is chosen for a competition, each batch chosen with probability \tfrac14, independently of all the yields. Yields have mean 30 gallons and standard deviation 5 gallons, independent. Find the mean and standard deviation of the total yield, the chosen batch included.

Prediction and first steps

  1. For the density of Exercise 4, find the mean squared error of predicting X by E(X \mid Y) and by the constant E(X). By what fraction does knowing Y reduce it?
  2. Barrels pass a tasting independently with probability \tfrac25. Carlos tastes until two consecutive barrels have the same result (both pass or both fail). Find the mean number tasted.

Proofs

  1. Let T \sim \operatorname{Geometric}(p) with 0 < p \le 1, and assume E(T^2) is finite. By conditioning on the first trial, prove \displaystyle E(T) = \frac1p and \displaystyle E(T^2) = \frac{2 - p}{p^2}.
  2. Let E(X^2) < \infty. Prove \operatorname{Var}(E(X \mid Y)) \le \operatorname{Var}(X), with equality exactly when X = E(X \mid Y) with probability 1.
  3. Let E(X^2) and E(Y^2) be finite. Prove \operatorname{Cov}(X, Y) = \operatorname{Cov}(E(X \mid Y), Y).
  4. Give X and Y with \operatorname{Cov}(X, Y) = 0 but E(X \mid Y) not constant.
  5. Let E(X^2) and E(Y^2) be finite, with E(X \mid Y) = Y and E(Y \mid X) = X with probability 1. Prove P(X = Y) = 1.
Answers
  1. E(X \mid X > 2) = 5 and \displaystyle E(X \mid X \le 2) = \frac{3 - 5e^{-2/3}}{1 - e^{-2/3}} \approx 0.8897. By parts, E(X \mathbf{1}_{\{X > 2\}}) = 5e^{-2/3}, divided by P(X > 2) = e^{-2/3}. Result 1 with Y = \mathbf{1}_{\{X > 2\}} gives 3 = 5e^{-2/3} + E(X \mid X \le 2)(1 - e^{-2/3}).
  2. E(X \mid Y = 0) = \tfrac12, E(X \mid Y = 1) = \tfrac{11}{8}, E(X \mid Y) = \tfrac12 + \tfrac78 Y, E(X) = \tfrac{13}{12}; E(Y \mid X = 0) = \tfrac13, E(Y \mid X = 1) = \tfrac35, E(Y \mid X = 2) = 1. The row totals are \tfrac13 and \tfrac23, the column totals \tfrac14, \tfrac{5}{12}, \tfrac13.
  3. \displaystyle E(M \mid X = x) = \frac{x^2 - x + 42}{12} and E(M) = \tfrac{161}{36}. Given X = x, M = x for the x second rolls up to x, and M = k for each k > x, which sum to \displaystyle 21 - \frac{x(x + 1)}{2}. Averaging, \displaystyle \frac{1}{72} \sum_{x=1}^{6} (x^2 - x + 42) = \tfrac{322}{72}.
  4. \displaystyle E(X \mid Y = y) = \frac{y}{2}, \displaystyle \operatorname{Var}(X \mid Y = y) = \frac{y^2}{12}, \displaystyle E(Y \mid X = x) = \frac{1 + 2x}{3}, E(X) = \tfrac14. f_Y(y) = 6y(1 - y), so given Y = y, X \sim \operatorname{Uniform}(0, y). f_X(x) = 3(1 - x)^2, so Y given X = x has density \displaystyle \frac{2(1 - y)}{(1 - x)^2} on (x, 1). E(Y) = \tfrac12.
  5. E(X) = \tfrac32, \operatorname{Var}(X) = \tfrac{15}{4} and P(X = 0) = \tfrac25. E(X \mid Y) = \operatorname{Var}(X \mid Y) = 3Y, with E(Y) = \tfrac12 and \operatorname{Var}(Y) = \tfrac14, so Result 3 gives \tfrac32 + \tfrac94. Result 1 gives P(X = 0) = E(e^{-3Y}) = \int_0^\infty 2e^{-5y} \, dy.
  6. E(X) = 1 day and \operatorname{Var}(X) = \tfrac53. \displaystyle E(X \mid Y) = \frac{Y}{2}, \displaystyle \operatorname{Var}(X \mid Y) = \frac{Y^2}{12}, E(Y) = 2, \operatorname{Var}(Y) = 4, E(Y^2) = 8, so \operatorname{Var}(X) = \tfrac{8}{12} + \tfrac44.
  7. E(N) = 25, \operatorname{Var}(N) = \tfrac{650}{3} \approx 216.67, P(N = 50) = \tfrac{1}{51}. E(\operatorname{Var}(N \mid P)) = 50 E(P - P^2) = \tfrac{25}{3} and \operatorname{Var}(50P) = \tfrac{2500}{12}; P(N = 50) = \int_0^1 p^{50} \, dp.
  8. E(N) = 2 \ln 2 \approx 1.3863, P(N = 1) = \tfrac34, P(N = 2) = \tfrac16. Average \displaystyle \frac1P, P and P(1 - P) against the density 2 on (\tfrac12, 1).
  9. Mean 320 pounds, standard deviation \displaystyle \sqrt{2632} \approx 51.30 pounds. E(N) = 8, \operatorname{Var}(N) = 1.6, and Result 5 gives 9 \cdot 8 + 1600 \cdot 1.6.
  10. Mean 120 gallons, standard deviation \displaystyle \sqrt{10{,}900} \approx 104.40 gallons. N \sim \operatorname{Geometric}(\tfrac14) has mean 4 and variance 12, and Result 5 gives 25 \cdot 4 + 900 \cdot 12.
  11. \tfrac{1}{40} and \tfrac{3}{80}; knowing Y cuts the error by one third. \displaystyle E(\operatorname{Var}(X \mid Y)) = \frac{E(Y^2)}{12} with E(Y^2) = \tfrac{3}{10}, and \operatorname{Var}(X) = \tfrac{1}{10} - \tfrac{1}{16}.
  12. \tfrac{56}{19} \approx 2.947. Let a and b be the mean number still to taste after a pass and after a fail. Conditioning on the next barrel, a = 1 + \tfrac35 b and b = 1 + \tfrac25 a, so a = \tfrac{40}{19}, b = \tfrac{35}{19}, and the answer is 1 + \tfrac25 a + \tfrac35 b.
  13. Let F be the event that trial 1 fails. On F^c, T = 1. On F, T = 1 + T', where T' counts trials from the second on, is \operatorname{Geometric}(p) and is independent of F. By Result 5 of Variance and covariance, E(T \mid F) = 1 + E(T) and E(T^2 \mid F) = 1 + 2E(T) + E(T^2). Result 1 with Y = \mathbf{1}_F gives E(T) = p + (1 - p)(1 + E(T)), so pE(T) = 1. Then E(T^2) = p + (1 - p)(1 + \tfrac2p + E(T^2)), so \displaystyle pE(T^2) = 1 + \frac{2(1 - p)}{p} = \frac{2 - p}{p}.
  14. By Result 3, \operatorname{Var}(X) exceeds \operatorname{Var}(E(X \mid Y)) by E(\operatorname{Var}(X \mid Y)) \ge 0. By Result 1, E(\operatorname{Var}(X \mid Y)) = E((X - m(Y))^2). So equality holds exactly when E((X - m(Y))^2) = 0, which by the lemma in the proof of Result 5 of Markov and Chebyshev inequalities means X = m(Y) with probability 1.
  15. XY has a finite mean (Result 3 of Variance and covariance), so Result 1 with h(y) = y gives E(XY) = E(Y m(Y)), and E(m(Y)) = E(X). By Result 3, m(Y) has finite variance, so \operatorname{Cov}(m(Y), Y) is E(Y m(Y)) - E(m(Y)) E(Y), which is E(XY) - E(X) E(Y), that is \operatorname{Cov}(X, Y).
  16. Let Y be -1, 0 or 1 with probability \tfrac13 each, and X = Y^2. Then E(Y) = E(Y^3) = 0, so \operatorname{Cov}(X, Y) = E(Y^3) - E(Y^2) E(Y) = 0. By Result 2(b), E(X \mid Y) = Y^2, which is 0 or 1, not constant.
  17. By Result 3 of Variance and covariance, XY has a finite mean. Result 1 with h(y) = y gives E(XY) = E(Y \cdot Y) = E(Y^2), and with the letters exchanged E(XY) = E(X^2). So E((X - Y)^2), which is E(X^2) - 2E(XY) + E(Y^2), is 0, and the lemma in the proof of Result 5 of Markov and Chebyshev inequalities gives X = Y with probability 1.
math.wiki · CC-BY-SA-4.0