Orthonormal bases and Gram-Schmidt, proved
In R^n, coordinates along perpendicular unit vectors are single dot products. Does every finite-dimensional inner product space, polynomial and complex spaces included, have such a basis, and what does having one buy?
Where you are 5 in Inner product spaces
Why this exists
The Gram-Schmidt process makes any basis of a subspace of \mathbb{R}^n orthonormal, and Orthogonal bases and projection onto a subspace shows that weights along an orthonormal basis are dot products. Those proofs used only what an inner product keeps, so they should work in \mathcal{P}_2(\mathbb{R}), C[0, 1] and \mathbb{C}^n. This lesson proves that they do, and collects what such a basis buys.
Yuna and Kofi run a hydrophone station that records the ocean in blocks of four pressure samples. They store each block as its coordinates along an orthonormal basis of \mathbb{R}^4, drop the smallest, and want the error without rebuilding the block (Example 3).
Definition
Let V be an inner product space over \mathbb{F}. A list (\mathbf{q}_1, \dots, \mathbf{q}_m) in V is orthonormal when each vector has norm 1 and is orthogonal to the others:
An orthonormal basis of V is an orthonormal list that is a basis of V. For a subset U of V, the orthogonal complement is
a subspace, since each map \mathbf{v} \mapsto \langle \mathbf{v}, \mathbf{u} \rangle is linear (Result 2 of Inner products and norms). For a finite-dimensional subspace U, Result 6 shows V = U \oplus U^{\perp} (Sums and direct sums of subspaces); the U-part of \mathbf{v} is the orthogonal projection \operatorname{proj}_U \mathbf{v}. A linear functional on V is a linear map \varphi: V \to \mathbb{F} (Dual spaces).
Intuition
The inner product with \mathbf{q}_k kills every other basis vector and reads off the k-th coordinate alone. In the figure, \mathbf{v} = (2, 1) has \langle \mathbf{v}, \mathbf{q}_1 \rangle = 2 and \langle \mathbf{v}, \mathbf{q}_2 \rangle = -1: the feet of the perpendiculars. In other bases a coordinate costs a linear system; here it costs one inner product.
Gram-Schmidt also keeps a nested chain of spans, and an upper-triangular matrix says exactly that such a chain is invariant under the operator. That is why triangular form survives the process (Schur’s theorem).
Results
Throughout, V is an inner product space over \mathbb{F}.
Result 1 (orthonormal lists). If (\mathbf{q}_1, \dots, \mathbf{q}_m) is orthonormal, then for all a_1, \dots, a_m \in \mathbb{F},
So every orthonormal list is linearly independent, and if \dim V = n, every orthonormal list of length n is an orthonormal basis of V.
Result 2 (Gram-Schmidt). Let (\mathbf{v}_1, \dots, \mathbf{v}_m) be linearly independent in V. Put \mathbf{q}_1 = \mathbf{v}_1 / \lVert \mathbf{v}_1 \rVert and, for k = 2, \dots, m,
Then every \mathbf{w}_k \neq \mathbf{0}, (\mathbf{q}_1, \dots, \mathbf{q}_m) is orthonormal, and for each k, \mathbf{q}_1, \dots, \mathbf{q}_k span the same subspace as \mathbf{v}_1, \dots, \mathbf{v}_k.
Result 3 (coordinates are inner products). Let (\mathbf{q}_1, \dots, \mathbf{q}_n) be an orthonormal basis of V. For all \mathbf{u}, \mathbf{v} \in V,
and Parseval’s identities hold:
Result 4 (orthonormal bases exist and extend). Every finite-dimensional inner product space has an orthonormal basis, and every orthonormal list in a finite-dimensional V extends to an orthonormal basis of V.
Result 5 (Schur’s theorem). Let V be finite-dimensional and T \in \mathcal{L}(V). If T has an upper-triangular matrix with respect to some basis of V, then it has one with respect to some orthonormal basis. In particular, every operator on a finite-dimensional complex inner product space has an upper-triangular matrix with respect to some orthonormal basis.
Result 6 (orthogonal complements and closest points). Let U be a finite-dimensional subspace of V with orthonormal basis (\mathbf{q}_1, \dots, \mathbf{q}_m). Then V = U \oplus U^{\perp}, and
It is the closest point of U to \mathbf{v}: \lVert \mathbf{v} - \operatorname{proj}_U \mathbf{v} \rVert \le \lVert \mathbf{v} - \mathbf{u} \rVert for every \mathbf{u} \in U, with equality only for \mathbf{u} = \operatorname{proj}_U \mathbf{v}. If V is finite-dimensional, then also \dim U^{\perp} = \dim V - \dim U and (U^{\perp})^{\perp} = U.
Result 7 (Riesz representation theorem). Let V be finite-dimensional and \varphi a linear functional on V. Exactly one \mathbf{u} \in V has \varphi(\mathbf{v}) = \langle \mathbf{v}, \mathbf{u} \rangle for every \mathbf{v} \in V, and for any orthonormal basis (\mathbf{q}_1, \dots, \mathbf{q}_n) of V,
Why it is true
Result 1. Expand with additivity in both slots and \langle \mathbf{u}, \lambda\mathbf{v} \rangle = \bar{\lambda}\langle \mathbf{u}, \mathbf{v} \rangle (Result 2 of Inner products and norms):
If \sum_k a_k\mathbf{q}_k = \mathbf{0}, then \sum_k |a_k|^2 = 0 and every a_k = 0: the list is independent. An independent list of length \dim V is a basis (Result 5 of Bases and dimension, proved).
Result 2. Induction on k. Independence gives \mathbf{v}_1 \neq \mathbf{0}, so \mathbf{q}_1 is a unit vector on the same line. Suppose (\mathbf{q}_1, \dots, \mathbf{q}_{k-1}) is orthonormal with the same span X_{k-1} as (\mathbf{v}_1, \dots, \mathbf{v}_{k-1}). If \mathbf{w}_k = \mathbf{0}, then \mathbf{v}_k is a combination of \mathbf{v}_1, \dots, \mathbf{v}_{k-1}, which independence forbids; so \mathbf{q}_k is a unit vector. For j < k, linearity in the first slot gives
So (\mathbf{q}_1, \dots, \mathbf{q}_k) is orthonormal. Each \mathbf{q}_i, i \le k, lies in X_k = \operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\} (\mathbf{w}_k is \mathbf{v}_k minus a vector of X_{k-1}), and both spans have dimension k (Result 1), so they are equal (Result 6 of Bases and dimension, proved). The coefficient is \langle \mathbf{v}_k, \mathbf{q}_i \rangle, not its conjugate, because of Result 5 of Inner products and norms; over \mathbb{C} the other order fails (Example 2).
Result 3. Write \mathbf{v} = \sum_j a_j\mathbf{q}_j. Then \langle \mathbf{v}, \mathbf{q}_k \rangle = \sum_j a_j\delta_{jk} = a_k. The norm formula is Result 1, and expanding \langle \mathbf{u}, \mathbf{v} \rangle with \mathbf{v} = \sum_k b_k\mathbf{q}_k, b_k = \langle \mathbf{v}, \mathbf{q}_k \rangle, gives the last.
Result 4. Let (\mathbf{q}_1, \dots, \mathbf{q}_m) be orthonormal, \dim V = n. It is independent (Result 1), so it extends to a basis of V by appending \mathbf{v}_{m+1}, \dots, \mathbf{v}_n (Result 3 of Bases and dimension, proved). Run Gram-Schmidt on it. The first m vectors come out unchanged: if \mathbf{q}_1, \dots, \mathbf{q}_{k-1} did, every coefficient \langle \mathbf{q}_k, \mathbf{q}_j \rangle with j < k is 0, so \mathbf{w}_k = \mathbf{q}_k. The output is orthonormal of length n, a basis by Result 1. With m = 0 this is existence, since a finite-dimensional space has a basis (Result 2 of Bases and dimension, proved).
Result 5. Column k of the matrix with respect to (\mathbf{v}_1, \dots, \mathbf{v}_n) holds the coordinates of T\mathbf{v}_k (The matrix of a linear map), so the matrix is upper triangular exactly when T\mathbf{v}_k \in \operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\} for each k, that is, when every \operatorname{span}\{\mathbf{v}_1, \dots, \mathbf{v}_k\} is invariant under T. Gram-Schmidt gives an orthonormal basis with the same spans (Result 2), so the same test passes for it. Over \mathbb{C}, every operator on a finite-dimensional space has an upper-triangular matrix with respect to some basis; that is stated here without proof and proved in Upper-triangular matrices over the complex numbers.
Result 6. Put \mathbf{p} = \sum_k \langle \mathbf{v}, \mathbf{q}_k \rangle \mathbf{q}_k \in U. By orthonormality \langle \mathbf{p}, \mathbf{q}_j \rangle = \langle \mathbf{v}, \mathbf{q}_j \rangle for each j, so \mathbf{v} - \mathbf{p} is orthogonal to each \mathbf{q}_j, hence to their span U (Result 2 of Inner products and norms). So V = U + U^{\perp}. A vector in U \cap U^{\perp} is orthogonal to itself, hence \mathbf{0}, so the sum is direct (Result 3 of Sums and direct sums of subspaces) and \operatorname{proj}_U \mathbf{v} = \mathbf{p}.
For \mathbf{u} \in U, \mathbf{v} - \mathbf{u} = (\mathbf{v} - \mathbf{p}) + (\mathbf{p} - \mathbf{u}) with the parts in U^{\perp} and U, so by the Pythagorean theorem
with equality only when \mathbf{u} = \mathbf{p}.
If \dim V = n, extend to an orthonormal basis (\mathbf{q}_1, \dots, \mathbf{q}_n) of V (Result 4). Then \mathbf{v} \in U^{\perp} exactly when \langle \mathbf{v}, \mathbf{q}_j \rangle = 0 for j \le m, which by Result 3 means \mathbf{v} \in \operatorname{span}\{\mathbf{q}_{m+1}, \dots, \mathbf{q}_n\}. So \dim U^{\perp} = n - m, and the same argument with the two parts of the basis swapped gives (U^{\perp})^{\perp} = \operatorname{span}\{\mathbf{q}_1, \dots, \mathbf{q}_m\} = U.
Result 7. Since \langle \mathbf{v}, \bar{c}\,\mathbf{q}_k \rangle = c\langle \mathbf{v}, \mathbf{q}_k \rangle, Result 3 and the linearity of \varphi give
If \mathbf{u}' also works, then \langle \mathbf{v}, \mathbf{u} - \mathbf{u}' \rangle = 0 for all \mathbf{v}; taking \mathbf{v} = \mathbf{u} - \mathbf{u}' gives \mathbf{u} = \mathbf{u}'. So \mathbf{u} depends on \varphi alone, not on the basis.
Worked examples
An orthonormal basis of quadratics
In \mathcal{P}_2(\mathbb{R}) with \langle p, q \rangle = \int_0^1 p(x)q(x)\,dx, apply Gram-Schmidt to (1, x, x^2).
SolutionThis is the integral inner product of C[0, 1] restricted to a subspace, so it is an inner product. Every integral comes from \displaystyle \int_0^1 x^n\,dx = \frac{1}{n + 1}.
Since \lVert 1 \rVert = 1, q_1 = 1. Next \displaystyle \langle x, 1 \rangle = \frac{1}{2}, so \displaystyle w_2 = x - \frac{1}{2}, with \displaystyle \lVert w_2 \rVert^2 = \frac{1}{12}, and \displaystyle q_2 = \sqrt{12}\left(x - \frac{1}{2}\right) = \sqrt{3}(2x - 1). For x^2: \displaystyle \langle x^2, 1 \rangle = \frac{1}{3} and \displaystyle \langle x^2, q_2 \rangle = \sqrt{3}\left(\frac{2}{4} - \frac{1}{3}\right) = \frac{\sqrt{3}}{6}, so
Expanding the square,
so \displaystyle q_3 = \sqrt{180}\,w_3 = \sqrt{5}(6x^2 - 6x + 1). The list (q_1, q_2, q_3) is orthonormal of length 3 = \dim \mathcal{P}_2(\mathbb{R}), an orthonormal basis (Result 1), and q_1, q_2 span \mathcal{P}_1(\mathbb{R}) (Result 2).
Complex scalars, and finishing the basis
In \mathbb{C}^3, apply Gram-Schmidt to \mathbf{v}_1 = (1, i, 0), \mathbf{v}_2 = (0, 1, i), extend the result to an orthonormal basis of \mathbb{C}^3, and find the coordinates of (0, 0, 1) in it.
Solution\lVert \mathbf{v}_1 \rVert^2 = 1 + |i|^2 = 2, so \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, i, 0). The second slot is conjugated: \displaystyle \langle \mathbf{v}_2, \mathbf{q}_1 \rangle = \frac{1}{\sqrt{2}}(1 \cdot \bar{i}) = -\frac{i}{\sqrt{2}}. Then
with \displaystyle \lVert \mathbf{w}_2 \rVert^2 = \frac{3}{2}, so \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{6}}(i, 1, 2i). Check: \langle \mathbf{q}_2, \mathbf{q}_1 \rangle is a multiple of i + 1 \cdot \bar{i} = 0. With the slots reversed, the coefficient would be \displaystyle \frac{i}{\sqrt{2}} and the result \displaystyle \left(-\frac{i}{2}, \frac{3}{2}, i\right), whose inner product with \mathbf{q}_1 is \displaystyle -\sqrt{2}\,i \neq 0.
To extend: \operatorname{span}\{\mathbf{q}_1, \mathbf{q}_2\}^{\perp} has dimension 1 (Result 6), so we need a unit vector on that line. \mathbf{z} \perp \mathbf{q}_1 means z_1 - iz_2 = 0, and \mathbf{z} \perp \mathbf{q}_2 means -iz_1 + z_2 - 2iz_3 = 0. Substituting z_1 = iz_2 gives 2z_2 - 2iz_3 = 0, so z_2 = iz_3, z_1 = -z_3, and \displaystyle \mathbf{q}_3 = \frac{1}{\sqrt{3}}(-1, i, 1).
By Result 3 the coordinates of \mathbf{e}_3 = (0, 0, 1) are 0, \displaystyle \frac{\overline{2i}}{\sqrt{6}} = -\frac{2i}{\sqrt{6}} and \displaystyle \frac{1}{\sqrt{3}}. Parseval checks it: \displaystyle \frac{4}{6} + \frac{1}{3} = 1 = \lVert \mathbf{e}_3 \rVert^2.
The hydrophone block
Yuna and Kofi use the orthonormal basis \displaystyle \mathbf{q}_1 = \frac{1}{2}(1, 1, 1, 1), \displaystyle \mathbf{q}_2 = \frac{1}{2}(1, 1, -1, -1), \displaystyle \mathbf{q}_3 = \frac{1}{\sqrt{2}}(1, -1, 0, 0), \displaystyle \mathbf{q}_4 = \frac{1}{\sqrt{2}}(0, 0, 1, -1) of \mathbb{R}^4. A block reads \mathbf{s} = (6, 4, 5, 9). Find its coordinates, drop the one of least absolute value, and find the error of the stored block before rebuilding it.
SolutionThe four vectors have norm 1 and pairwise dot products 0, so they are an orthonormal basis (Result 1). By Result 3 the coordinates are dot products:
Parseval: 144 + 4 + 2 + 8 = 158 = 36 + 16 + 25 + 81. Dropping the \mathbf{q}_3 term leaves the error \displaystyle \sqrt{2}\,\mathbf{q}_3, of norm \displaystyle \sqrt{2} by Result 1. Rebuilt, the block is \mathbf{s} - (1, -1, 0, 0) = (5, 5, 5, 9). It is \operatorname{proj}_U \mathbf{s} for U = \operatorname{span}\{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_4\}, so by Result 6 no vector of U is closer to \mathbf{s}. Dropping \mathbf{q}_2, \mathbf{q}_4 or \mathbf{q}_1 instead would cost 2, \displaystyle 2\sqrt{2} or 12.
The best line for a square root
Find the p \in \mathcal{P}_1(\mathbb{R}) closest to \displaystyle f(x) = \sqrt{x} in C[0, 1], and the distance \lVert f - p \rVert.
SolutionBy Example 1, \displaystyle \left(1, \sqrt{3}(2x - 1)\right) is an orthonormal basis of U = \mathcal{P}_1(\mathbb{R}), and by Result 6 the answer is \operatorname{proj}_U f. With \displaystyle \langle \sqrt{x}, 1 \rangle = \frac{2}{3} and \displaystyle \langle \sqrt{x}, x \rangle = \frac{2}{5},
so \displaystyle p = \frac{2}{3} + \frac{2}{5}(2x - 1), that is, \displaystyle p = \frac{4}{5}x + \frac{4}{15}. Since f - p \perp p, the Pythagorean theorem and Result 1 give
so \displaystyle \lVert f - p \rVert = \frac{1}{15\sqrt{2}} \approx 0.0471.
Evaluation as an inner product
On \mathcal{P}_2(\mathbb{R}) with the inner product of Example 1, find the polynomial u with p(0) = \langle p, u \rangle for every p.
SolutionEvaluation at 0 is linear, so Result 7 applies, and all values are real. With q_1(0) = 1, \displaystyle q_2(0) = -\sqrt{3}, \displaystyle q_3(0) = \sqrt{5}:
Check on 1, x, x^2: \int_0^1 u\,dx = 10 - 18 + 9 = 1, \displaystyle \int_0^1 xu\,dx = \frac{15}{2} - 12 + \frac{9}{2} = 0 and \int_0^1 x^2u\,dx = 6 - 9 + 3 = 0, their values at 0.
Matrix entries are inner products
Let \mathcal{B} = (\mathbf{q}_1, \dots, \mathbf{q}_n) be an orthonormal basis of V and T \in \mathcal{L}(V). Prove that the entry in row j, column k of [T]_{\mathcal{B}} is \langle T\mathbf{q}_k, \mathbf{q}_j \rangle.
SolutionColumn k of [T]_{\mathcal{B}} is [T\mathbf{q}_k]_{\mathcal{B}} (The matrix of a linear map). By Result 3, T\mathbf{q}_k = \sum_j \langle T\mathbf{q}_k, \mathbf{q}_j \rangle \mathbf{q}_j, and coordinates in a basis are unique (Result 1 of Bases and dimension, proved), so the j-th coordinate is \langle T\mathbf{q}_k, \mathbf{q}_j \rangle.
A triangular form that cannot be diagonal
Let T\mathbf{x} = A\mathbf{x} on \mathbb{R}^2 with the dot product, A = \begin{bmatrix} 2 & 1 \\ -1 & 4 \end{bmatrix}. Find an orthonormal basis in which T has an upper-triangular matrix, and the matrix.
Solution\det(A - \lambda I) = (2 - \lambda)(4 - \lambda) + 1, which is (\lambda - 3)^2 (The characteristic equation and finding eigenvalues), and A - 3I has null space spanned by \mathbf{v}_1 = (1, 1). With \mathbf{v}_2 = (0, 1), both \operatorname{span}\{\mathbf{v}_1\} and \mathbb{R}^2 are invariant, so the matrix in (\mathbf{v}_1, \mathbf{v}_2) is upper triangular. Gram-Schmidt, as in Result 5: \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, 1), \displaystyle \mathbf{w}_2 = (0, 1) - \frac{1}{2}(1, 1), so \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{2}}(-1, 1).
By Example 6 the entries are inner products. A\mathbf{q}_1 = 3\mathbf{q}_1 gives the first column. \displaystyle A\mathbf{q}_2 = \frac{1}{\sqrt{2}}(-1, 5), so \displaystyle \langle A\mathbf{q}_2, \mathbf{q}_1 \rangle = \frac{-1 + 5}{2} = 2 and \displaystyle \langle A\mathbf{q}_2, \mathbf{q}_2 \rangle = \frac{1 + 5}{2} = 3:
No basis at all makes it diagonal: the only eigenvalue is 3 and its eigenspace is a line (Diagonalization).
Boundaries
- Result 6 needs U finite-dimensional. In C[0, 1] let U = \{f : f(0) = 0\}. If g \in U^{\perp}, then xg(x) \in U, so \int_0^1 xg(x)^2\,dx = 0; the continuous, nonnegative integrand is then 0 (as in Result 1 of Inner products and norms), so g = 0 on (0, 1], and at 0 by continuity. So U^{\perp} = \{0\} and U \oplus U^{\perp} = U \neq C[0, 1].
- Result 7 needs V finite-dimensional. On \mathcal{P}(\mathbb{R}) with the inner product of Example 1, p \mapsto p(0) has no representing polynomial (Exercise 17), though on \mathcal{P}_2(\mathbb{R}) it has one (Example 5).
- Gram-Schmidt needs independence. On a dependent list, \mathbf{w}_k = \mathbf{0} at the first \mathbf{v}_k in the span of the earlier ones (The Gram-Schmidt process).
- Over \mathbb{R}, Schur’s theorem needs its hypothesis. The rotation (x, y) \mapsto (-y, x) of \mathbb{R}^2 has no eigenvector, and the first column of an upper-triangular matrix would give one.
Where this goes next
Result 7 defines Adjoints and self-adjoint operators: \mathbf{v} \mapsto \langle T\mathbf{v}, \mathbf{w} \rangle is a linear functional, represented by a vector T^{*}\mathbf{w}; with Example 6, in an orthonormal basis T^{*} has the conjugate transpose matrix. The The spectral theorem asks when Schur’s triangular matrix is diagonal. In Fourier series, the scaled constant, sines and cosines form an infinite orthonormal list, and Result 6’s projections onto its first m vectors are the partial sums.
Exercises
Coordinates and Gram-Schmidt
- Show that \displaystyle \frac{1}{7}(2, 6, 3), \displaystyle \frac{1}{7}(3, 2, -6), \displaystyle \frac{1}{7}(6, -3, 2) form an orthonormal basis of \mathbb{R}^3, find the coordinates of (7, 0, 14), and check Parseval’s identity.
- In \mathbb{C}^2, apply Gram-Schmidt to (1 + i, 1), (1, i).
- On \mathcal{P}_2(\mathbb{R}) take \langle p, q \rangle = p(-1)q(-1) + p(0)q(0) + p(1)q(1). Apply Gram-Schmidt to (1, x, x^2).
- In C[0, 1], apply Gram-Schmidt to (x, x^2) in that order.
Projections and closest points
- In C[-\pi, \pi], show that \displaystyle \frac{1}{\sqrt{\pi}}\sin x, \displaystyle \frac{1}{\sqrt{\pi}}\cos x, \displaystyle \frac{1}{\sqrt{\pi}}\sin 2x are orthonormal. Project f(x) = x onto their span and find the squared distance from f to the span.
- Find the polynomial in \mathcal{P}_1(\mathbb{R}) closest to x^3 in C[0, 1], and the distance.
- With the basis of Example 3, a block reads (2, 6, 7, 5). Keep its two coordinates of largest absolute value. What is the stored block, and what is the error?
- Find the distance from x^2 to \mathcal{P}_1(\mathbb{R}) in C[0, 1], and the closest point.
Riesz and Schur
In Exercises 9 and 10, \mathcal{P}_2(\mathbb{R}) has the inner product of Example 1.
- Find u with p(1) = \langle p, u \rangle for every p \in \mathcal{P}_2(\mathbb{R}).
- Find u with p'(0) = \langle p, u \rangle for every p \in \mathcal{P}_2(\mathbb{R}).
- Find an orthonormal basis of \mathbb{R}^2 in which \mathbf{x} \mapsto A\mathbf{x}, A = \begin{bmatrix} 1 & 1 \\ -4 & 5 \end{bmatrix}, has an upper-triangular matrix, and give the matrix.
- On \mathbb{C}^2, find an orthonormal basis in which \mathbf{z} \mapsto A\mathbf{z}, A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}, has an upper-triangular matrix. What do you notice?
Proofs
- Prove the converse of Result 1: if its identity holds for all scalars a_1, \dots, a_m, then (\mathbf{q}_1, \dots, \mathbf{q}_m) is orthonormal.
- (Bessel’s inequality.) Let (\mathbf{q}_1, \dots, \mathbf{q}_m) be orthonormal in any inner product space V. Prove \sum_k |\langle \mathbf{v}, \mathbf{q}_k \rangle|^2 \le \lVert \mathbf{v} \rVert^2, with equality exactly when \mathbf{v} \in \operatorname{span}\{\mathbf{q}_1, \dots, \mathbf{q}_m\}.
- For subsets U \subseteq W of V, prove W^{\perp} \subseteq U^{\perp}. For subspaces U, W, prove (U + W)^{\perp} = U^{\perp} \cap W^{\perp}.
- Let (\mathbf{q}_1, \dots, \mathbf{q}_n) be an orthonormal basis of V, and let C\mathbf{v} \in \mathbb{F}^n have entries \langle \mathbf{v}, \mathbf{q}_k \rangle. Prove that C is an isomorphism V \to \mathbb{F}^n with \langle C\mathbf{u}, C\mathbf{v} \rangle = \langle \mathbf{u}, \mathbf{v} \rangle, the left side Euclidean.
- On \mathcal{P}(\mathbb{R}) with \langle p, q \rangle = \int_0^1 p(x)q(x)\,dx, prove that no u \in \mathcal{P}(\mathbb{R}) has p(0) = \langle p, u \rangle for every p.
- For a finite-dimensional subspace U of V, prove that \operatorname{proj}_U is a linear operator on V with \operatorname{proj}_U^2 = \operatorname{proj}_U, range U and null space U^{\perp}.
Answers
- The pairwise dot products are \displaystyle \frac{6 + 12 - 18}{49}, \displaystyle \frac{12 - 18 + 6}{49}, \displaystyle \frac{18 - 6 - 12}{49}, all 0, and each squared norm is 1; by Result 1 this is a basis. Coordinates: \displaystyle \frac{14 + 42}{7} = 8, \displaystyle \frac{21 - 84}{7} = -9, \displaystyle \frac{42 + 28}{7} = 10. Parseval: 64 + 81 + 100 = 245 = 49 + 196.
- \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{3}}(1 + i, 1). Then \displaystyle \langle \mathbf{v}_2, \mathbf{q}_1 \rangle = \frac{(1 - i) + i}{\sqrt{3}} = \frac{1}{\sqrt{3}}, so \displaystyle \mathbf{w}_2 = \frac{1}{3}(2 - i, -1 + 3i), with \displaystyle \lVert \mathbf{w}_2 \rVert^2 = \frac{5 + 10}{9} = \frac{5}{3}, and \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{15}}(2 - i, -1 + 3i).
- It is an inner product: \langle p, p \rangle = 0 gives p three roots, so p = 0 (The remainder and factor theorems). The values at -1, 0, 1 are (1, 1, 1), (-1, 0, 1), (1, 0, 1). \displaystyle q_1 = \frac{1}{\sqrt{3}}; \langle x, 1 \rangle = 0, so \displaystyle q_2 = \frac{x}{\sqrt{2}}. \displaystyle \langle x^2, q_1 \rangle = \frac{2}{\sqrt{3}} and \langle x^2, q_2 \rangle = 0, so \displaystyle w_3 = x^2 - \frac{2}{3}, with values \displaystyle \left(\frac{1}{3}, -\frac{2}{3}, \frac{1}{3}\right) and \displaystyle \lVert w_3 \rVert^2 = \frac{2}{3}. So \displaystyle q_3 = \frac{3x^2 - 2}{\sqrt{6}}.
- \displaystyle \lVert x \rVert^2 = \frac{1}{3}, so \displaystyle q_1 = \sqrt{3}\,x. \displaystyle \langle x^2, q_1 \rangle = \frac{\sqrt{3}}{4}, so \displaystyle w_2 = x^2 - \frac{3}{4}x, with \displaystyle \lVert w_2 \rVert^2 = \frac{1}{5} - \frac{3}{8} + \frac{3}{16} = \frac{1}{80}, and \displaystyle q_2 = \sqrt{5}(4x^2 - 3x).
- Each of \sin^2 x, \cos^2 x, \sin^2 2x integrates to \pi over [-\pi, \pi], and the cross products integrate to 0. \langle x, \sin x \rangle = 2\pi, \langle x, \cos x \rangle = 0, \langle x, \sin 2x \rangle = -\pi, so the projection is 2\sin x - \sin 2x. The squared distance is \lVert x \rVert^2 - 5\pi, that is, \displaystyle \frac{2\pi^3}{3} - 5\pi \approx 4.96.
- \displaystyle \langle x^3, 1 \rangle = \frac{1}{4} and \displaystyle \langle x^3, \sqrt{3}(2x - 1) \rangle = \frac{3\sqrt{3}}{20}, so the closest point is \displaystyle \frac{1}{4} + \frac{9}{20}(2x - 1) = \frac{9}{10}x - \frac{1}{5}. Squared distance \displaystyle \frac{1}{7} - \frac{1}{16} - \frac{27}{400} = \frac{9}{700}, distance \displaystyle \frac{3}{10\sqrt{7}} \approx 0.113.
- Coordinates: 10, -2, \displaystyle -2\sqrt{2}, \displaystyle \sqrt{2}. Keeping 10 and \displaystyle -2\sqrt{2} stores \displaystyle 10\mathbf{q}_1 - 2\sqrt{2}\,\mathbf{q}_3 = (3, 7, 5, 5), with error \displaystyle \sqrt{(-2)^2 + (\sqrt{2})^2} = \sqrt{6}. Check: (-1, -1, 2, 0) has norm \displaystyle \sqrt{6}.
- With Example 1’s basis, x^2 - \operatorname{proj}_{\mathcal{P}_1} x^2 = \langle x^2, q_3 \rangle q_3, and \displaystyle \langle x^2, q_3 \rangle = \sqrt{5}\left(\frac{6}{5} - \frac{6}{4} + \frac{1}{3}\right) = \frac{\sqrt{5}}{30}. The distance is \displaystyle \frac{\sqrt{5}}{30} = \frac{1}{6\sqrt{5}}, and the closest point is \displaystyle x^2 - \frac{1}{6}(6x^2 - 6x + 1) = x - \frac{1}{6}.
- q_1(1) = 1, \displaystyle q_2(1) = \sqrt{3}, \displaystyle q_3(1) = \sqrt{5}, so u = 1 + 3(2x - 1) + 5(6x^2 - 6x + 1), which is 30x^2 - 24x + 3. Check: \int_0^1 u, \int_0^1 xu, \int_0^1 x^2u are all 1.
- q_1' = 0, \displaystyle q_2'(0) = 2\sqrt{3}, \displaystyle q_3'(0) = -6\sqrt{5}, so u = 6(2x - 1) - 30(6x^2 - 6x + 1), which is -180x^2 + 192x - 36. Check: \int_0^1 u = 0, \int_0^1 xu = 1, \int_0^1 x^2u = 0.
- \det(A - \lambda I) = (\lambda - 3)^2, and A - 3I has null space spanned by (1, 2). Take \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{5}}(1, 2), \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{5}}(-2, 1). Then A\mathbf{q}_1 = 3\mathbf{q}_1 and \displaystyle A\mathbf{q}_2 = \frac{1}{\sqrt{5}}(-1, 13), so by Example 6 the matrix is \begin{bmatrix} 3 & 5 \\ 0 & 3 \end{bmatrix} (with -\mathbf{q}_2 the 5 becomes -5).
- A(1, -i) = (i, 1) = i(1, -i), so take \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, -i) and \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{2}}(1, i). Since A(1, i) = -i(1, i), the matrix is \begin{bmatrix} i & 0 \\ 0 & -i \end{bmatrix}: diagonal, not just triangular. Over \mathbb{R} this A has no triangular matrix at all (Boundaries); the The spectral theorem explains the diagonal.
- a_j = 1 and the rest 0 gives \lVert \mathbf{q}_j \rVert^2 = 1. For j \neq k, take a_j = 1, a_k = c, the rest 0. By Result 3 of Inner products and norms,
which the hypothesis makes 1 + |c|^2. With c = 1, \operatorname{Re}\langle \mathbf{q}_j, \mathbf{q}_k \rangle = 0; over \mathbb{C}, c = i gives \operatorname{Im}\langle \mathbf{q}_j, \mathbf{q}_k \rangle = 0. 14. U = \operatorname{span}\{\mathbf{q}_1, \dots, \mathbf{q}_m\} has orthonormal basis (\mathbf{q}_1, \dots, \mathbf{q}_m). With \mathbf{p} = \operatorname{proj}_U \mathbf{v}, Result 6 and the Pythagorean theorem give \lVert \mathbf{v} \rVert^2 = \lVert \mathbf{p} \rVert^2 + \lVert \mathbf{v} - \mathbf{p} \rVert^2, which is at least \lVert \mathbf{p} \rVert^2 = \sum_k |\langle \mathbf{v}, \mathbf{q}_k \rangle|^2 (Result 1). Equality holds exactly when \mathbf{v} = \mathbf{p}, which forces \mathbf{v} \in U; conversely, if \mathbf{v} \in U, then \mathbf{v} = \mathbf{p} by Result 3 applied in U. 15. A vector orthogonal to all of W is orthogonal to all of U, so W^{\perp} \subseteq U^{\perp}. As U, W \subseteq U + W, this gives (U + W)^{\perp} \subseteq U^{\perp} \cap W^{\perp}. Conversely, if \mathbf{v} \perp U and \mathbf{v} \perp W, additivity in the second slot gives \langle \mathbf{v}, \mathbf{u} + \mathbf{w} \rangle = 0 + 0 = 0 for \mathbf{u} \in U, \mathbf{w} \in W. 16. C is linear by linearity in the first slot. If C\mathbf{v} = \mathbf{0}, then \mathbf{v} = \mathbf{0} by Result 3, so C is injective; by orthonormality C sends a_1\mathbf{q}_1 + \dots + a_n\mathbf{q}_n to (a_1, \dots, a_n), so C is surjective, hence an isomorphism (One-to-one, onto and bijective functions). The Euclidean inner product of C\mathbf{u} and C\mathbf{v} is \sum_k \langle \mathbf{u}, \mathbf{q}_k \rangle \overline{\langle \mathbf{v}, \mathbf{q}_k \rangle}, which is \langle \mathbf{u}, \mathbf{v} \rangle by Parseval. 17. Suppose u exists. Then p = xu has p(0) = 0, so \int_0^1 xu(x)^2\,dx = 0. The integrand is continuous and nonnegative, so it is 0 on [0, 1] (were it positive at a point, it would stay above half that value on an interval, making the integral positive). So u vanishes on (0, 1], has infinitely many roots, and is the zero polynomial (The remainder and factor theorems). But then p = 1 gives 1 = \langle 1, 0 \rangle = 0, a contradiction. 18. Take an orthonormal basis (\mathbf{q}_1, \dots, \mathbf{q}_m) of U (Result 4). By Result 6, \operatorname{proj}_U \mathbf{v} = \sum_k \langle \mathbf{v}, \mathbf{q}_k \rangle \mathbf{q}_k, linear in \mathbf{v} by linearity in the first slot, with values in U. Each \mathbf{u} \in U equals \operatorname{proj}_U \mathbf{u} by Result 3 in U, so the range is U and \operatorname{proj}_U^2 = \operatorname{proj}_U. Finally \operatorname{proj}_U \mathbf{v} = \mathbf{0} exactly when every \langle \mathbf{v}, \mathbf{q}_k \rangle = 0 (the \mathbf{q}’s are independent), that is, when \mathbf{v} is orthogonal to a spanning list of U, which means \mathbf{v} \in U^{\perp} by additivity and conjugate homogeneity in the second slot.