The spectral theorem
A symmetric matrix always seems to come with perpendicular eigenvectors. Which operators on an inner product space have an orthonormal basis made of eigenvectors, and why exactly those?
Where you are 5 in Inner product spaces
Why this exists
Diagonalization stated without proof that every real symmetric matrix can be diagonalized by perpendicular unit eigenvectors, and Extrema and the second derivative test classified critical points with that fact. When an operator is diagonalizable says when a basis of eigenvectors exists. An inner product space asks for more: an orthonormal basis of eigenvectors.
Schur’s theorem (Result 5 of Orthonormal bases and Gram-Schmidt, proved) gives an orthonormal basis in which the matrix is upper triangular. When can that triangle be made diagonal? Over \mathbb{C}, exactly for the normal operators; over \mathbb{R}, exactly for the self-adjoint ones of Adjoints and self-adjoint operators.
Definition
Let V be a finite-dimensional inner product space over \mathbb{F} and T \in \mathcal{L}(V). T is orthogonally diagonalizable (over \mathbb{C} also unitarily diagonalizable) when some orthonormal basis of V consists entirely of eigenvectors of T; equivalently, T has a diagonal matrix with respect to some orthonormal basis. The set of eigenvalues of T is its spectrum, which gives the theorem its name.
A square matrix Q \in \mathbb{F}^{n \times n} is unitary when Q^{*}Q = I; a real one with Q^{T}Q = I is orthogonal. Entry (j, k) of Q^{*}Q is \langle \mathbf{q}_k, \mathbf{q}_j \rangle for the columns \mathbf{q}_1, \dots, \mathbf{q}_n, so Q^{*}Q = I says exactly that the columns are orthonormal in \mathbb{F}^n. Such a Q is invertible with Q^{-1} = Q^{*} (The invertible matrix theorem).
For an eigenvalue \lambda of T, write P_{\lambda} = \operatorname{proj}_{E_\lambda} for the orthogonal projection onto its eigenspace.
Intuition
In an orthonormal eigenbasis (\mathbf{q}_1, \dots, \mathbf{q}_n) with T\mathbf{q}_k = \lambda_k\mathbf{q}_k, every vector is \mathbf{v} = \sum_k \langle \mathbf{v}, \mathbf{q}_k \rangle \mathbf{q}_k, and T multiplies the kth coordinate by \lambda_k. For a real symmetric 2 \times 2 matrix this is visible: the unit circle goes to an ellipse whose axes are the eigenvector lines.
Normality is forced: if [T]_{\mathcal{B}} = D is diagonal in an orthonormal basis, so is [T^{*}]_{\mathcal{B}} = \bar{D}, and diagonal matrices commute. The theorem says this is also enough.
Results
Throughout, V is a finite-dimensional inner product space.
Result 1 (self-adjoint operators on real spaces have eigenvalues). If V is a nonzero real inner product space and T \in \mathcal{L}(V) is self-adjoint, then T has an eigenvalue.
Result 2 (the real spectral theorem). Let \mathbb{F} = \mathbb{R} and T \in \mathcal{L}(V). The following are equivalent: (a) T is self-adjoint; (b) V has an orthonormal basis consisting of eigenvectors of T; (c) T has a diagonal matrix with respect to some orthonormal basis of V.
Result 3 (the complex spectral theorem). Let \mathbb{F} = \mathbb{C} and T \in \mathcal{L}(V). The following are equivalent: (a) T is normal; (b) V has an orthonormal basis consisting of eigenvectors of T; (c) T has a diagonal matrix with respect to some orthonormal basis of V.
Result 4 (spectral decomposition). Let T be self-adjoint (if \mathbb{F} = \mathbb{R}) or normal (if \mathbb{F} = \mathbb{C}), with distinct eigenvalues \lambda_1, \dots, \lambda_m, and write P_j = P_{\lambda_j}. Then the eigenspaces are pairwise orthogonal, V = E_{\lambda_1} \oplus \dots \oplus E_{\lambda_m}, and for every polynomial p,
In particular T = \lambda_1P_1 + \dots + \lambda_mP_m and I = P_1 + \dots + P_m.
Result 5 (matrix form). A matrix A \in \mathbb{R}^{n \times n} is symmetric if and only if A = QDQ^{T} for an orthogonal Q and a real diagonal D. A matrix A \in \mathbb{C}^{n \times n} is normal if and only if A = QDQ^{*} for a unitary Q and a diagonal D, and then A is Hermitian if and only if D is real. In each case AQ = QD: the columns of Q are orthonormal eigenvectors and the diagonal of D lists their eigenvalues.
Why it is true
Result 1. Choose an orthonormal basis \mathcal{B} of V (Result 4 of Orthonormal bases and Gram-Schmidt, proved) and let A = [T]_{\mathcal{B}}, a real n \times n matrix, n \ge 1. By Result 3 of Adjoints and self-adjoint operators, A^{T} = A^{*} = A. So A acting on \mathbb{C}^n with the Euclidean inner product is self-adjoint. Every operator on a nonzero finite-dimensional complex space has an eigenvalue (Upper-triangular matrices over the complex numbers), and this one’s eigenvalue \lambda is real by Result 6 of Adjoints and self-adjoint operators. Take an eigenvector \mathbf{z} = \mathbf{x} + i\mathbf{y} with \mathbf{x}, \mathbf{y} \in \mathbb{R}^n, not both zero. Then A\mathbf{x} + iA\mathbf{y} = \lambda\mathbf{x} + i\lambda\mathbf{y}, and as A and \lambda are real, comparing real and imaginary parts gives A\mathbf{x} = \lambda\mathbf{x} and A\mathbf{y} = \lambda\mathbf{y}. Whichever of \mathbf{x}, \mathbf{y} is nonzero is [\mathbf{v}]_{\mathcal{B}} for some \mathbf{v} \neq \mathbf{0}, and [T\mathbf{v}]_{\mathcal{B}} = A[\mathbf{v}]_{\mathcal{B}} = \lambda[\mathbf{v}]_{\mathcal{B}}, so T\mathbf{v} = \lambda\mathbf{v}.
The step both theorems share. Let T be normal, \mathbf{q} a unit vector with T\mathbf{q} = \lambda\mathbf{q}, and W = (\operatorname{span}\{\mathbf{q}\})^{\perp}. Then W is invariant under T and T^{*}, and S = T|_W is normal; if T is self-adjoint, so is S. Indeed T^{*}\mathbf{q} = \bar{\lambda}\mathbf{q} by Result 7 of Adjoints and self-adjoint operators, so for \mathbf{w} \in W,
For \mathbf{w}, \mathbf{w}' \in W, \langle S\mathbf{w}, \mathbf{w}' \rangle = \langle \mathbf{w}, T^{*}\mathbf{w}' \rangle with T^{*}\mathbf{w}' \in W, so by the uniqueness in Result 1 of Adjoints and self-adjoint operators, applied in W, S^{*} = T^{*}|_W. Then SS^{*} and S^{*}S are the restrictions of TT^{*} = T^{*}T to W; and if T^{*} = T, then S^{*} = T|_W = S.
Results 2 and 3. (b) and (c) are equivalent: the matrix with respect to (\mathbf{q}_1, \dots, \mathbf{q}_n) is diagonal exactly when each T\mathbf{q}_k is a multiple of the nonzero \mathbf{q}_k.
(c) implies (a): [T^{*}]_{\mathcal{B}} = \bar{D} by Result 3 of Adjoints and self-adjoint operators, and the matrix of a product is the product of the matrices (The matrix of a linear map), so [TT^{*}]_{\mathcal{B}} = D\bar{D} = \bar{D}D = [T^{*}T]_{\mathcal{B}} and T is normal. Over \mathbb{R}, \bar{D} = D, so T^{*} = T.
(a) implies (b), by induction on n = \dim V. For n = 0 the empty list works. For n \ge 1, T has an eigenvalue: over \mathbb{C} by Upper-triangular matrices over the complex numbers, over \mathbb{R} by Result 1. Scale an eigenvector to a unit vector \mathbf{q}_1 and let W = (\operatorname{span}\{\mathbf{q}_1\})^{\perp}, of dimension n - 1 since V = \operatorname{span}\{\mathbf{q}_1\} \oplus W (Result 6 of Orthonormal bases and Gram-Schmidt, proved). By the shared step, T|_W is normal (self-adjoint over \mathbb{R}), so by the induction hypothesis W has an orthonormal basis (\mathbf{q}_2, \dots, \mathbf{q}_n) of eigenvectors of T|_W, hence of T. Each lies in W, so (\mathbf{q}_1, \dots, \mathbf{q}_n) is orthonormal, hence linearly independent (Result 1 of Orthonormal bases and Gram-Schmidt, proved), and an independent list of length \dim V is a basis (Bases and dimension, proved).
Result 4. Eigenvectors for distinct eigenvalues are orthogonal by Result 7 of Adjoints and self-adjoint operators. Grouping an orthonormal eigenbasis (Result 2 or 3) by eigenvalue writes every \mathbf{v} as \mathbf{u}_1 + \dots + \mathbf{u}_m with \mathbf{u}_j \in E_{\lambda_j}. The writing is unique: if \mathbf{u}_1 + \dots + \mathbf{u}_m = \mathbf{0}, the inner product with \mathbf{u}_k gives \lVert \mathbf{u}_k \rVert^2 = 0. So the sum is direct (Sums and direct sums of subspaces). As \mathbf{v} - \mathbf{u}_j is orthogonal to E_{\lambda_j}, the unique splitting of Result 6 of Orthonormal bases and Gram-Schmidt, proved gives \mathbf{u}_j = P_j\mathbf{v}. Hence I = \sum_j P_j and T = \sum_j \lambda_jP_j. Also P_jP_k = 0 for j \neq k (the range of P_k is orthogonal to E_{\lambda_j}) and P_j^2 = P_j, so T^k = \sum_j \lambda_j^kP_j by induction on k, and linear combinations give p(T).
Result 5. If A is real symmetric, \mathbf{x} \mapsto A\mathbf{x} on \mathbb{R}^n with the dot product is self-adjoint (its matrix in the orthonormal standard basis is A = A^{T}). Result 2 gives orthonormal \mathbf{q}_1, \dots, \mathbf{q}_n with A\mathbf{q}_k = \lambda_k\mathbf{q}_k. With these as the columns of Q and D = \operatorname{diag}(\lambda_1, \dots, \lambda_n), Q^{T}Q = I and AQ = QD column by column, so A = QDQ^{T}. Conversely (QDQ^{T})^{T} = QD^{T}Q^{T} = QDQ^{T}. Over \mathbb{C} the first half is the same with Result 3; conversely (QDQ^{*})(QDQ^{*})^{*} = QDD^{*}Q^{*} equals (QDQ^{*})^{*}(QDQ^{*}) = QD^{*}DQ^{*}. Finally (QDQ^{*})^{*} = QD^{*}Q^{*} equals QDQ^{*} exactly when D^{*} = D (multiply by Q^{*} on the left and Q on the right), that is, when D is real.
Worked examples
A repeated eigenvalue in three dimensions
Let A = \begin{bmatrix} 3 & 2 & 2 \\ 2 & 6 & 4 \\ 2 & 4 & 6 \end{bmatrix}. Find an orthogonal Q and a diagonal D with A = QDQ^{T}.
SolutionA is symmetric, so Result 5 promises them. Subtracting 2I leaves \mathbf{u}\mathbf{u}^{T} for \mathbf{u} = (1, 2, 2), so A = 2I + \mathbf{u}\mathbf{u}^{T}. Then A\mathbf{u} = 2\mathbf{u} + \mathbf{u}(\mathbf{u}^{T}\mathbf{u}) = 11\mathbf{u}, and E_2 = \ker \mathbf{u}\mathbf{u}^{T} is the plane x_1 + 2x_2 + 2x_3 = 0, of dimension 2. Eigenspaces for distinct eigenvalues form a direct sum, and 1 + 2 = 3 already fills \mathbb{R}^3, so 11 and 2 are the only eigenvalues.
The plane has the basis (-2, 1, 0), (-2, 0, 1), which is not orthogonal (dot product 4), so apply Gram-Schmidt inside E_2:
\mathbf{u} is orthogonal to the plane by Result 4. Normalizing (1, 2, 2), (-2, 1, 0) and (-2, -4, 5) by 3, \displaystyle \sqrt{5} and \displaystyle 3\sqrt{5} gives
with D = \operatorname{diag}(11, 2, 2). Any orthonormal basis of the plane would do for the last two columns.
A normal matrix with complex eigenvalues
Let A = \begin{bmatrix} 2 + i & -1 - 2i \\ 1 + 2i & 2 + i \end{bmatrix} act on \mathbb{C}^2. Show that A is normal but not Hermitian, and find a unitary Q and diagonal D with A = QDQ^{*}.
SolutionA^{*} = \begin{bmatrix} 2 - i & 1 - 2i \\ -1 + 2i & 2 - i \end{bmatrix}. Multiplying out both ways,
so A is normal; its diagonal entries are not real, so A \neq A^{*}. Since \det A = (2 + i)^2 + (1 + 2i)^2 = 8i, the The characteristic equation and finding eigenvalues is \lambda^2 - (4 + 2i)\lambda + 8i = 0, which factors as (\lambda - 4)(\lambda - 2i) = 0. Row reducing gives
These are orthogonal, as Result 7 of Adjoints and self-adjoint operators requires: 1 \cdot 1 + i \cdot \overline{(-i)} = 1 + i^2 = 0. Each has norm \displaystyle \sqrt{2}, so
A normal triangular matrix is diagonal
Prove that an upper-triangular normal matrix A \in \mathbb{C}^{n \times n} is diagonal, and deduce (a) implies (c) of Result 3 from Schur’s theorem.
SolutionCompare the diagonal entries of A^{*}A and AA^{*}. Entry (k, k) of A^{*}A is \sum_i |a_{ik}|^2, the squared length of column k; entry (k, k) of AA^{*} is \sum_j |a_{kj}|^2, that of row k. We show by Strong induction and well-ordering on k that row k has no nonzero entry off the diagonal. Suppose this holds for rows 1, \dots, k - 1 (for k = 1 there is nothing to assume). Then the entries a_{ik} with i < k vanish, and those with i > k vanish because A is upper triangular, so column k has squared length |a_{kk}|^2. Row k has squared length |a_{kk}|^2 + \sum_{j > k} |a_{kj}|^2. Normality makes the two equal, so \sum_{j > k} |a_{kj}|^2 = 0 and each a_{kj} with j > k is 0.
Now let T be normal on a complex V. Schur’s theorem gives an orthonormal basis \mathcal{B} with A = [T]_{\mathcal{B}} upper triangular, and [T^{*}]_{\mathcal{B}} = A^{*} by Result 3 of Adjoints and self-adjoint operators, so AA^{*} = A^{*}A. By the first part A is diagonal.
Self-adjoint with a matrix that is not symmetric
On \mathbb{R}^2 take \langle \mathbf{x}, \mathbf{y} \rangle = x_1y_1 + 2x_2y_2 and T\mathbf{x} = A\mathbf{x} with A = \begin{bmatrix} 1 & 4 \\ 2 & 3 \end{bmatrix}. Show that T is self-adjoint and find an orthonormal basis of eigenvectors.
SolutionExpanding,
Swapping \mathbf{x} and \mathbf{y} leaves this unchanged, so it equals \langle A\mathbf{y}, \mathbf{x} \rangle = \langle \mathbf{x}, A\mathbf{y} \rangle, and T is self-adjoint although A \neq A^{T}. (Result 3 of Adjoints and self-adjoint operators needs an orthonormal basis, and here \displaystyle \lVert \mathbf{e}_2 \rVert = \sqrt{2}.)
The characteristic equation \lambda^2 - 4\lambda - 5 = 0 gives 5 with eigenvector (1, 1) and -1 with eigenvector (2, -1). Their dot product is 1, but \langle (1, 1), (2, -1) \rangle = 2 - 2 = 0. With \lVert (1, 1) \rVert^2 = 3 and \lVert (2, -1) \rVert^2 = 6, the orthonormal eigenbasis is
A spectral decomposition and a square root
Let A = \begin{bmatrix} 8 & 2 \\ 2 & 5 \end{bmatrix}. Write A = \lambda_1P_1 + \lambda_2P_2, give a formula for A^k, and find a symmetric R with nonnegative eigenvalues and R^2 = A.
SolutionThe characteristic equation \lambda^2 - 13\lambda + 36 = 0 gives 9 and 4, with unit eigenvectors \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{5}}(2, 1) and \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{5}}(1, -2). The projection onto the line through a unit \mathbf{q} is \mathbf{x} \mapsto (\mathbf{x} \cdot \mathbf{q})\mathbf{q} = \mathbf{q}\mathbf{q}^{T}\mathbf{x} (Orthogonal bases and projection onto a subspace), so
Indeed \displaystyle 9P_1 + 4P_2 = \frac{1}{5}\begin{bmatrix} 40 & 10 \\ 10 & 25 \end{bmatrix} = A and P_1 + P_2 = I. Result 4 with p(z) = z^k gives A^k = 9^kP_1 + 4^kP_2. For the root, any p with p(9) = 3 and p(4) = 2 gives R = p(A) = 3P_1 + 2P_2, and R^2 = 9P_1 + 4P_2 = A because P_1P_2 = P_2P_1 = 0 and P_j^2 = P_j:
R is symmetric with eigenvalues 3 and 2.
The cobbler’s measurements
Yuna and Kofi run a cobbler’s shop. For each customer they record how far foot length and foot width fall from the shop’s averages, in millimeters, and the covariance matrix of the two measurements is C = \begin{bmatrix} 10 & 3 \\ 3 & 2 \end{bmatrix} (in \text{mm}^2). For a unit vector \mathbf{u}, the combined measurement along \mathbf{u} has variance \mathbf{u}^{T}C\mathbf{u}. In which direction is the variance largest, and in which smallest? Prove the answer.
SolutionC is symmetric. Its characteristic equation \lambda^2 - 12\lambda + 11 = 0 gives 11 and 1, with orthogonal unit eigenvectors \displaystyle \mathbf{q}_1 = \frac{1}{\sqrt{10}}(3, 1) and \displaystyle \mathbf{q}_2 = \frac{1}{\sqrt{10}}(1, -3). Any unit \mathbf{u} is c_1\mathbf{q}_1 + c_2\mathbf{q}_2 with c_1^2 + c_2^2 = 1 (Result 1 of Orthonormal bases and Gram-Schmidt, proved), and C\mathbf{u} = 11c_1\mathbf{q}_1 + c_2\mathbf{q}_2, so
This runs from 1 to 11: the largest variance, 11\ \text{mm}^2, is along \pm\mathbf{q}_1, and the smallest, 1\ \text{mm}^2, along \pm\mathbf{q}_2. In general the maximum over unit vectors is the largest eigenvalue, as Lagrange multipliers found by calculus.
Boundaries
- Over \mathbb{R}, normal is not enough. The rotation R(x, y) = (-y, x) of \mathbb{R}^2 has R^{*} = -R, so RR^{*} = R^{*}R = I, but it has no eigenvector: R\mathbf{v} = \lambda\mathbf{v} would give \lambda\lVert \mathbf{v} \rVert^2 = \langle R\mathbf{v}, \mathbf{v} \rangle = 0, so \lambda = 0, while R is injective. Over \mathbb{C} the same matrix has an orthonormal eigenbasis (Exercise 10).
- Diagonalizable is not enough. B = \begin{bmatrix} 1 & 1 \\ 0 & 2 \end{bmatrix} on \mathbb{R}^2 has eigenvectors only along (1, 0) and (1, 1), which are not orthogonal. For a different inner product B becomes self-adjoint (Exercise 18).
- Finite dimension is used. On C[0, 1] with \langle f, g \rangle = \int_0^1 f(x)g(x)\,dx, the map (Mf)(x) = xf(x) satisfies \langle Mf, g \rangle = \langle f, Mg \rangle, yet has no eigenvalue: xf(x) = \lambda f(x) forces f(x) = 0 for x \neq \lambda, so f = 0 by Continuity.
Where this goes next
The Singular value decomposition applies the spectral theorem to T^{*}T, which is self-adjoint with eigenvalues at least 0 (Example 5 of Adjoints and self-adjoint operators), to find orthonormal bases suited to any linear map. For operators that are not normal, Generalized eigenvectors and Jordan canonical form replace eigenspaces by larger invariant subspaces that still split V as a direct sum.
Exercises
Orthonormal eigenbases
- Find an orthogonal Q and a diagonal D with A = QDQ^{T} for A = \begin{bmatrix} 1 & 6 \\ 6 & -4 \end{bmatrix}.
- Do the same for A = \begin{bmatrix} 4 & 0 & -1 \\ 0 & 4 & 1 \\ -1 & 1 & 5 \end{bmatrix}.
- Find a unitary Q and a real diagonal D with H = QDQ^{*} for H = \begin{bmatrix} 0 & 1 - i \\ 1 + i & 1 \end{bmatrix}.
- Let P(z_1, z_2, z_3) = (z_3, z_1, z_2) on \mathbb{C}^3 and \omega = e^{2\pi i/3}. Show that P is normal and that (1, \omega, \omega^2) is an eigenvector, and find an orthonormal basis of eigenvectors.
- On \mathcal{P}_2(\mathbb{R}) with \langle p, q \rangle = \int_{-1}^{1} p(x)q(x)\,dx, let (Tp)(x) = \big((1 - x^2)p'(x)\big)'. Show that T is self-adjoint and find an orthonormal basis of eigenvectors with their eigenvalues.
Decompositions and extreme values
- Write S = \begin{bmatrix} 5 & -4 \\ -4 & 5 \end{bmatrix} as \lambda_1P_1 + \lambda_2P_2, and find a symmetric R with nonnegative eigenvalues and R^2 = S.
- Use Result 4 to compute M^4 for M = \begin{bmatrix} 0 & 2 \\ 2 & 3 \end{bmatrix}.
- Find the largest and smallest values of 11x^2 + 24xy + 18y^2 on the circle x^2 + y^2 = 1, and where they occur.
- Find the real symmetric 3 \times 3 matrix whose eigenspace for 2 is \operatorname{span}\{(1, 1, 1)\} and whose eigenspace for -1 is the plane orthogonal to (1, 1, 1).
Testing the hypotheses
- With the Euclidean inner product, decide which of these has an orthonormal basis of eigenvectors, and give one when it does: (a) \begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix} on \mathbb{R}^2; (b) \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} on \mathbb{C}^2; (c) \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} on \mathbb{C}^2; (d) \begin{bmatrix} 2 & i \\ i & 2 \end{bmatrix} on \mathbb{C}^2.
- For which real b does \begin{bmatrix} 1 & b \\ 2 & 1 \end{bmatrix} have an orthonormal basis of eigenvectors in \mathbb{R}^2, and for which in \mathbb{C}^2?
Proofs
In these, V is a finite-dimensional inner product space.
- Let V be complex and T normal. Prove that \langle T\mathbf{v}, \mathbf{v} \rangle is real and at least 0 for every \mathbf{v} if and only if every eigenvalue of T is real and at least 0.
- Let V be complex and T normal. Prove that T^{*}T = I if and only if every eigenvalue of T has absolute value 1.
- Let V be real and T self-adjoint with T^3 = I. Prove that T = I.
- Let T be self-adjoint with \langle T\mathbf{v}, \mathbf{v} \rangle \ge 0 for every \mathbf{v}. Prove that there is a self-adjoint R with \langle R\mathbf{v}, \mathbf{v} \rangle \ge 0 for every \mathbf{v} and R^2 = T.
- Let T be normal on a complex V \neq \{\mathbf{0}\}. Prove that the largest value of \lVert T\mathbf{v} \rVert over unit vectors \mathbf{v} is the largest absolute value of an eigenvalue of T.
- Let V be real and S, T self-adjoint with ST = TS. Prove that V has an orthonormal basis of vectors that are eigenvectors of both.
- Let V be a finite-dimensional real vector space and T \in \mathcal{L}(V) diagonalizable. Prove that some inner product on V makes T self-adjoint.
Answers
- The characteristic equation \lambda^2 + 3\lambda - 40 = 0 gives 5 with eigenvector (3, 2) and -8 with (2, -3), orthogonal. So \displaystyle Q = \frac{1}{\sqrt{13}}\begin{bmatrix} 3 & 2 \\ 2 & -3 \end{bmatrix} and D = \operatorname{diag}(5, -8).
- The characteristic polynomial is -(\lambda - 3)(\lambda - 4)(\lambda - 6). Row reduction gives (1, -1, 1) for 3, (1, 1, 0) for 4 and (1, -1, -2) for 6, pairwise orthogonal. So Q has columns \displaystyle \frac{1}{\sqrt{3}}(1, -1, 1), \displaystyle \frac{1}{\sqrt{2}}(1, 1, 0), \displaystyle \frac{1}{\sqrt{6}}(1, -1, -2), and D = \operatorname{diag}(3, 4, 6).
- H^{*} = H. The characteristic equation \lambda^2 - \lambda - 2 = 0 gives 2 with eigenvector (1, 1 + i) and -1 with (-1 + i, 1); their inner product is (-1 - i) + (1 + i) = 0 and each has norm \displaystyle \sqrt{3}. So \displaystyle Q = \frac{1}{\sqrt{3}}\begin{bmatrix} 1 & -1 + i \\ 1 + i & 1 \end{bmatrix} and D = \operatorname{diag}(2, -1).
- The matrix of P permutes the standard basis, so its columns are orthonormal and P^{*}P = PP^{*} = I. Using \omega^3 = 1, P(1, \omega, \omega^2) = (\omega^2, 1, \omega) = \omega^2(1, \omega, \omega^2). Likewise (1, 1, 1) has eigenvalue 1 and (1, \omega^2, \omega) has eigenvalue \omega. As \bar{\omega} = \omega^2 and 1 + \omega + \omega^2 = 0, all three inner products vanish, and each vector has norm \displaystyle \sqrt{3}: the basis is \displaystyle \frac{1}{\sqrt{3}}(1, 1, 1), \displaystyle \frac{1}{\sqrt{3}}(1, \omega, \omega^2), \displaystyle \frac{1}{\sqrt{3}}(1, \omega^2, \omega).
- Integrating by parts, the boundary term vanishes because 1 - x^2 = 0 at \pm 1, so \langle Tp, q \rangle = -\int_{-1}^{1} (1 - x^2)p'(x)q'(x)\,dx, which is symmetric in p and q. Then T1 = 0, Tx = -2x and T(3x^2 - 1) = -6(3x^2 - 1). Normalizing, the basis is \displaystyle \frac{1}{\sqrt{2}} (eigenvalue 0), \displaystyle \frac{\sqrt{6}}{2}x (eigenvalue -2), \displaystyle \frac{\sqrt{10}}{4}(3x^2 - 1) (eigenvalue -6).
- Eigenvalue 9 with (1, -1) and 1 with (1, 1), so \displaystyle P_1 = \frac{1}{2}\begin{bmatrix} 1 & -1 \\ -1 & 1 \end{bmatrix}, \displaystyle P_2 = \frac{1}{2}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} and S = 9P_1 + P_2. Then R = 3P_1 + P_2 = \begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}.
- Eigenvalue 4 with (1, 2) and -1 with (2, -1), so M^4 = 4^4P_1 + (-1)^4P_2 with \displaystyle P_1 = \frac{1}{5}\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix} and \displaystyle P_2 = \frac{1}{5}\begin{bmatrix} 4 & -2 \\ -2 & 1 \end{bmatrix}, which gives M^4 = \begin{bmatrix} 52 & 102 \\ 102 & 205 \end{bmatrix}.
- The form is \mathbf{x}^{T}A\mathbf{x} with A = \begin{bmatrix} 11 & 12 \\ 12 & 18 \end{bmatrix}, whose eigenvalues are 27 with (3, 4) and 2 with (4, -3). As in Example 6, the largest value is 27 at \displaystyle \pm(\frac{3}{5}, \frac{4}{5}) and the smallest is 2 at \displaystyle \pm(\frac{4}{5}, -\frac{3}{5}).
- With \displaystyle P = \frac{1}{3}J the projection onto \operatorname{span}\{(1, 1, 1)\} (J all ones), Result 4 gives 2P - (I - P) = 3P - I = \begin{bmatrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{bmatrix}.
- (a) No: the eigenvectors lie along (1, 0) and (1, 1), not orthogonal. (b) Yes: A^{*} = -A, so A is normal; \displaystyle \frac{1}{\sqrt{2}}(1, -i) for i and \displaystyle \frac{1}{\sqrt{2}}(1, i) for -i. (c) No: AA^{*} has (1, 1) entry 2 and A^{*}A has 1, so it is not normal (its eigenvectors all lie along (1, 0)). (d) Yes: AA^{*} = A^{*}A = 5I, and \displaystyle \frac{1}{\sqrt{2}}(1, 1) for 2 + i, \displaystyle \frac{1}{\sqrt{2}}(1, -1) for 2 - i.
- In \mathbb{R}^2 the matrix must be symmetric (Result 2): b = 2. In \mathbb{C}^2 it must be normal (Result 3). The off-diagonal entries of AA^{T} and A^{T}A always agree and their (1, 1) entries are 1 + b^2 and 5, so b = 2 or b = -2 (eigenvalues 1 \pm 2i).
- Suppose \langle T\mathbf{v}, \mathbf{v} \rangle is real and at least 0 for every \mathbf{v}. If T\mathbf{v} = \lambda\mathbf{v} with \mathbf{v} \neq \mathbf{0}, then \lambda\lVert \mathbf{v} \rVert^2 = \langle T\mathbf{v}, \mathbf{v} \rangle, so \lambda is real and at least 0. Conversely, Result 3 gives an orthonormal basis (\mathbf{q}_1, \dots, \mathbf{q}_n) with T\mathbf{q}_k = \lambda_k\mathbf{q}_k, and each \lambda_k is an eigenvalue, so real and at least 0. For \mathbf{v} = \sum_k c_k\mathbf{q}_k, orthonormality gives \langle T\mathbf{v}, \mathbf{v} \rangle = \sum_k \lambda_kc_k\bar{c}_k = \sum_k \lambda_k|c_k|^2 \ge 0.
- Result 3 gives an orthonormal basis \mathcal{B} = (\mathbf{q}_1, \dots, \mathbf{q}_n) with T\mathbf{q}_k = \lambda_k\mathbf{q}_k, so [T]_{\mathcal{B}} = D = \operatorname{diag}(\lambda_1, \dots, \lambda_n) and [T^{*}]_{\mathcal{B}} = \bar{D} (Result 3 of Adjoints and self-adjoint operators). The \lambda_k are all the eigenvalues: if T\mathbf{v} = \lambda\mathbf{v} with \mathbf{v} = \sum c_k\mathbf{q}_k \neq \mathbf{0}, then \sum c_k(\lambda_k - \lambda)\mathbf{q}_k = \mathbf{0}, so \lambda = \lambda_k for some k with c_k \neq 0. Now [T^{*}T]_{\mathcal{B}} = \bar{D}D = \operatorname{diag}(|\lambda_1|^2, \dots, |\lambda_n|^2), which is I exactly when every |\lambda_k| = 1.
- By Result 2, V has an orthonormal basis with T\mathbf{q}_k = \lambda_k\mathbf{q}_k, \lambda_k real. Then \mathbf{q}_k = T^3\mathbf{q}_k = \lambda_k^3\mathbf{q}_k, so \lambda_k^3 = 1, and the only real cube root of 1 is 1. So T\mathbf{q}_k = \mathbf{q}_k on a basis, and T = I.
- Take an orthonormal eigenbasis with T\mathbf{q}_k = \lambda_k\mathbf{q}_k (Result 2 or 3); then \lambda_k = \langle T\mathbf{q}_k, \mathbf{q}_k \rangle \ge 0. Define R by \displaystyle R\mathbf{q}_k = \sqrt{\lambda_k}\,\mathbf{q}_k. Its matrix is real and diagonal, so [R^{*}] = [R] and R is self-adjoint; R^2\mathbf{q}_k = \lambda_k\mathbf{q}_k = T\mathbf{q}_k on a basis, so R^2 = T; and for \mathbf{v} = \sum c_k\mathbf{q}_k, \displaystyle \langle R\mathbf{v}, \mathbf{v} \rangle = \sum_k \sqrt{\lambda_k}\,|c_k|^2 \ge 0.
- Take an orthonormal eigenbasis with T\mathbf{q}_k = \lambda_k\mathbf{q}_k, and let M = \max_k |\lambda_k|. For \mathbf{v} = \sum c_k\mathbf{q}_k, Result 1 of Orthonormal bases and Gram-Schmidt, proved gives \lVert T\mathbf{v} \rVert^2 = \sum |\lambda_k|^2|c_k|^2 \le M^2\sum |c_k|^2 = M^2\lVert \mathbf{v} \rVert^2. So \lVert T\mathbf{v} \rVert \le M for unit \mathbf{v}, with equality at a \mathbf{q}_k with |\lambda_k| = M.
- By Result 4, V = E_{\lambda_1} \oplus \dots \oplus E_{\lambda_m} for the eigenspaces of T, pairwise orthogonal. Each E_{\lambda_j} is invariant under S: if T\mathbf{v} = \lambda_j\mathbf{v}, then T(S\mathbf{v}) = S(T\mathbf{v}) = \lambda_jS\mathbf{v}. The restriction of S to E_{\lambda_j} is self-adjoint, so Result 2 gives an orthonormal basis of E_{\lambda_j} of eigenvectors of S, which are eigenvectors of T as they lie in E_{\lambda_j}. Joined together, these lists are orthonormal (different eigenspaces are orthogonal) and have length \sum_j \dim E_{\lambda_j} = \dim V, so they form a basis.
- Let (\mathbf{v}_1, \dots, \mathbf{v}_n) be a basis with T\mathbf{v}_k = \lambda_k\mathbf{v}_k, \lambda_k real. Define \langle \sum a_k\mathbf{v}_k, \sum b_k\mathbf{v}_k \rangle = \sum a_kb_k, the dot product of coordinate vectors. Taking coordinates is linear and injective, so positivity, symmetry and linearity in the first slot carry over from the dot product: this is an inner product. The basis is orthonormal for it and [T] = \operatorname{diag}(\lambda_1, \dots, \lambda_n) is real and diagonal, so (c) implies (a) of Result 2 makes T self-adjoint.