Linear Algebra

The transpose and symmetric matrices

A table of distances between huts reads the same down a column as along the matching row. What operation on a matrix captures that, and what does it do to sums, products and dot products?

You need

Matrix multiplicationPrecalculus

Unlocks

Least squares: the best approximate solution of Ax = bLinear Algebra

Where you are 12 in Matrices

Linear algebra6

Matrices12

▴ 3 above Matrix multiplication Rows, columns and dimensions

The transpose and symmetric matrices

What the determinant means Writing a system as a matrix equation ▾ Show all 12 (4 more)

Why this exists

Gwenaelle keeps the hiking club’s table of trail distances, in kilometers, between four mountain huts. Row i, column j holds the distance from hut i to hut j:

D = \begin{bmatrix} 0 & 7 & 12 & 9 \\ 7 & 0 & 6 & 11 \\ 12 & 6 & 0 & 8 \\ 9 & 11 & 8 & 0 \end{bmatrix}.

The trail from hut 1 to hut 3 is the trail from hut 3 to hut 1, so d_{13} = d_{31} = 12. Row 1 read across and column 1 read down hold the same four numbers, so swapping rows with columns changes nothing. Naming that swap pays off well beyond tables: a dot product is a row times a column, so writing \mathbf{u} \cdot \mathbf{v} as a matrix product needs a way to turn a column into a row, and the best approximate solution of an unsolvable system comes from the matrix A^T A (Least squares: the best approximate solution of Ax = b).

Definition

An m \times n matrix has m rows and n columns, and a_{ij} is its entry in row i, column j (Rows, columns and dimensions).

The transpose of an m \times n matrix A is the n \times m matrix A^T whose rows are the columns of A, in order. Entry by entry,

(A^T)_{ij} = a_{ji} \quad \text{for } 1 \le i \le n,\ 1 \le j \le m.

The transpose of a column vector in \mathbb{R}^n (an n \times 1 matrix) is a 1 \times n row, and the transpose of a row is a column.

A matrix S is symmetric when S^T = S. Since S^T is n \times m when S is m \times n, a symmetric matrix must be square, and a square matrix is symmetric exactly when

s_{ij} = s_{ji} \quad \text{for every } i \text{ and } j.

The main diagonal is s_{11}, s_{22}, \dots, s_{nn}. The condition puts no demand on it; it pairs each entry off the diagonal with its mirror image across it.

Intuition

Turn the matrix over about its main diagonal, the line running down from the top-left corner. Entry (i, j) lands at position (j, i), and a short, wide matrix becomes a tall, narrow one.

Figure 1Row 1 of A becomes column 1 of A^T, and row 2 becomes column 2. A 2 \times 3 matrix turns into a 3 \times 2 one.

A symmetric matrix is one the flip leaves unchanged. Gwenaelle’s table is symmetric because distance does not depend on the direction of travel. Its zero diagonal comes from the meaning of distance, not from symmetry.

Figure 2A symmetric table. Each entry off the diagonal has a twin in the mirror position across the shaded diagonal.

Results

Result 1 (flipping twice). (A^T)^T = A.

Result 2 (sums and multiples). If A and B have the same size and c is a scalar, then (A + B)^T = A^T + B^T and (cA)^T = cA^T.

Result 3 (products reverse). If A is m \times n and B is n \times p, then

(AB)^T = B^T A^T.

Result 4 (dot products). For columns \mathbf{u}, \mathbf{v} in \mathbb{R}^n, \mathbf{u}^T \mathbf{v} is the 1 \times 1 matrix whose entry is \mathbf{u} \cdot \mathbf{v}. Reading a 1 \times 1 matrix as its entry, \mathbf{u} \cdot \mathbf{v} = \mathbf{u}^T \mathbf{v} and \lVert \mathbf{v} \rVert^2 = \mathbf{v}^T \mathbf{v}. For an m \times n matrix A, \mathbf{x} in \mathbb{R}^n and \mathbf{y} in \mathbb{R}^m,

(A\mathbf{x}) \cdot \mathbf{y} = \mathbf{x} \cdot (A^T \mathbf{y}).

Result 5 (symmetric from any matrix). For every matrix A, of any size, A^T A and A A^T are symmetric. Entry (i, j) of A^T A is column i of A dotted with column j; entry (i, j) of A A^T is row i dotted with row j. For square A, A + A^T is symmetric too.

Result 6 (inverses). If A is square and invertible, then A^T is invertible and (A^T)^{-1} = (A^{-1})^T.

Why it is true

Result 1: entry (i, j) of (A^T)^T is entry (j, i) of A^T, which is a_{ij}. Result 2: entry (i, j) of (A + B)^T is a_{ji} + b_{ji}, which is entry (i, j) of A^T + B^T; multiples work the same way.

Result 3. Sizes first: (AB)^T is p \times m, and B^T A^T is (p \times n)(n \times m), also p \times m. By the row-times-column rule (Matrix multiplication), entry (i, j) of (AB)^T is entry (j, i) of AB, and entry (i, j) of B^T A^T is row i of B^T times column j of A^T:

\sum_{k=1}^{n} a_{jk} b_{ki} \quad \text{and} \quad \sum_{k=1}^{n} b_{ki} a_{jk}.

Multiplication of numbers is commutative, so the sums agree term by term.

Result 4. The product of the 1 \times n matrix \mathbf{u}^T and the n \times 1 matrix \mathbf{v} has the single entry u_1 v_1 + \dots + u_n v_n = \mathbf{u} \cdot \mathbf{v}. Then Result 3 and the associative property of matrix multiplication give

(A\mathbf{x}) \cdot \mathbf{y} = (A\mathbf{x})^T \mathbf{y} = \mathbf{x}^T A^T \mathbf{y} = \mathbf{x} \cdot (A^T \mathbf{y}).

Result 5. By Results 3 and 1, (A^T A)^T = A^T (A^T)^T = A^T A, and likewise for A A^T. Column i of A is row i of A^T, which gives the dot-product reading. For square A, (A + A^T)^T = A^T + A by Results 2 and 1.

Result 6. Transpose A A^{-1} = I and A^{-1} A = I. By Result 3 they become (A^{-1})^T A^T = I and A^T (A^{-1})^T = I, since I^T = I.

Worked examples

Example 1

Reading a transpose

Find the transpose of P = \begin{bmatrix} 2 & -1 & 0 & 7 \\ 5 & 3 & -4 & 1 \end{bmatrix} and of the column \mathbf{x} = \begin{bmatrix} 3 \\ -2 \\ 6 \end{bmatrix}. Name the entries (P^T)_{31} and (P^T)_{42}.

Solution

P is 2 \times 4, so P^T is 4 \times 2. Its first column is the first row of P and its second column is the second row:

P^T = \begin{bmatrix} 2 & 5 \\ -1 & 3 \\ 0 & -4 \\ 7 & 1 \end{bmatrix}.

The column \mathbf{x} is 3 \times 1, so \mathbf{x}^T = \begin{bmatrix} 3 & -2 & 6 \end{bmatrix} is 1 \times 3. By the definition, (P^T)_{31} = p_{13} = 0 and (P^T)_{42} = p_{24} = 1, as the matrix above shows.

Example 2

Completing the hut table

Soizic measured trails between four huts and wrote the distances, in kilometers, into a table with gaps:

	hut 1	hut 2	hut 3	hut 4
hut 1	0	5	14	10
hut 2	5	0		8
hut 3		9	0
hut 4	10		3	0

Fill the gaps. How many distances must the club measure for n huts?

Solution

A distance does not depend on the direction of travel, so d_{ij} = d_{ji}, and each gap copies its filled mirror entry: row 2, column 3 copies d_{32} = 9; row 3, column 1 copies d_{13} = 14; row 3, column 4 copies d_{43} = 3; row 4, column 2 copies d_{24} = 8. So

D = \begin{bmatrix} 0 & 5 & 14 & 10 \\ 5 & 0 & 9 & 8 \\ 14 & 9 & 0 & 3 \\ 10 & 8 & 3 & 0 \end{bmatrix}.

The diagonal is 0, and the n^2 - n entries off it come in mirror pairs, one measurement per pair. The club needs \tfrac{n(n-1)}{2} distances; for four huts that is 6.

Example 3

The transpose of a product

Let

A = \begin{bmatrix} 1 & 0 & -2 \\ 3 & 5 & 4 \end{bmatrix}, \qquad B = \begin{bmatrix} 2 & 1 \\ 0 & -1 \\ 6 & 3 \end{bmatrix}.

Compute (AB)^T directly and as B^T A^T. Explain why A^T B^T cannot be the answer.

Solution

A is 2 \times 3 and B is 3 \times 2, so AB is 2 \times 2. Row 1 of A against the columns of B gives 2 + 0 - 12 = -10 and 1 + 0 - 6 = -5; row 2 gives 6 + 0 + 24 = 30 and 3 - 5 + 12 = 10. So

AB = \begin{bmatrix} -10 & -5 \\ 30 & 10 \end{bmatrix}, \qquad (AB)^T = \begin{bmatrix} -10 & 30 \\ -5 & 10 \end{bmatrix}.

Now B^T = \begin{bmatrix} 2 & 0 & 6 \\ 1 & -1 & 3 \end{bmatrix} and A^T = \begin{bmatrix} 1 & 3 \\ 0 & 5 \\ -2 & 4 \end{bmatrix}. Row 1 of B^T against the columns of A^T gives 2 + 0 - 12 = -10 and 6 + 0 + 24 = 30; row 2 gives 1 + 0 - 6 = -5 and 3 - 5 + 12 = 10. So B^T A^T equals (AB)^T, as Result 3 says.

The product A^T B^T is (3 \times 2)(2 \times 3), a 3 \times 3 matrix. It exists, but it cannot equal the 2 \times 2 matrix (AB)^T.

Example 4

Row times column, column times row

Let \mathbf{u} = \begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix} and \mathbf{v} = \begin{bmatrix} 4 \\ 0 \\ 1 \end{bmatrix}. Compute \mathbf{u}^T \mathbf{v}, \mathbf{u}^T \mathbf{u} and \mathbf{u} \mathbf{v}^T.

Solution

\mathbf{u}^T is 1 \times 3 and \mathbf{v} is 3 \times 1, so \mathbf{u}^T \mathbf{v} = 2(4) + (-1)(0) + 3(1) = 11, which is \mathbf{u} \cdot \mathbf{v} (Dot product, length and orthogonality in n dimensions). Likewise \mathbf{u}^T \mathbf{u} = 4 + 1 + 9 = 14 = \lVert \mathbf{u} \rVert^2.

The other order is (3 \times 1)(1 \times 3), a 3 \times 3 matrix with entry (i, j) equal to u_i v_j:

\mathbf{u} \mathbf{v}^T = \begin{bmatrix} 8 & 0 & 2 \\ -4 & 0 & -1 \\ 12 & 0 & 3 \end{bmatrix}.

The transpose on the left gives a number; on the right, a whole matrix.

Example 5

Building symmetric matrices from any matrix

For C = \begin{bmatrix} 1 & 2 \\ 0 & 1 \\ 3 & -1 \end{bmatrix}, compute C^T C and C C^T.

Solution

By Result 5, entry (i, j) of C^T C is column i of C dotted with column j. The columns \mathbf{c}_1 = (1, 0, 3) and \mathbf{c}_2 = (2, 1, -1) give \mathbf{c}_1 \cdot \mathbf{c}_1 = 10, \mathbf{c}_1 \cdot \mathbf{c}_2 = 2 + 0 - 3 = -1 and \mathbf{c}_2 \cdot \mathbf{c}_2 = 6, so

C^T C = \begin{bmatrix} 10 & -1 \\ -1 & 6 \end{bmatrix}.

For C C^T use the rows (1, 2), (0, 1) and (3, -1): squared lengths 5, 1, 10 and pairwise dot products 2, 3 - 2 = 1 and -1:

C C^T = \begin{bmatrix} 5 & 2 & 1 \\ 2 & 1 & -1 \\ 1 & -1 & 10 \end{bmatrix}.

Both are symmetric because a dot product does not depend on the order of its vectors, and their diagonal entries, being squared lengths, are never negative.

Example 6

A symmetric matrix with unknowns

Find every x and y for which

S = \begin{bmatrix} 1 & x + 2 & 5 \\ 3x - 4 & 0 & y \\ 2y - 9 & 7 & 6 \end{bmatrix}

is symmetric, and write S.

Solution

S is square, so it is symmetric exactly when its three mirror pairs agree. Positions (1, 2) and (2, 1) give x + 2 = 3x - 4, so x = 3. Positions (2, 3) and (3, 2) give y = 7. Positions (1, 3) and (3, 1) give 5 = 2y - 9, so y = 7 again. Three equations in two unknowns could have clashed; they agree, so x = 3, y = 7 is the only solution, and

S = \begin{bmatrix} 1 & 5 & 5 \\ 5 & 0 & 7 \\ 5 & 7 & 6 \end{bmatrix}.

Example 7

When is a product of symmetric matrices symmetric?

Show that for symmetric n \times n matrices S and T, the product ST is symmetric exactly when ST = TS. Then test S = \begin{bmatrix} 1 & 2 \\ 2 & 0 \end{bmatrix} and T = \begin{bmatrix} 0 & 1 \\ 1 & 3 \end{bmatrix}.

Solution

By Result 3 and S^T = S, T^T = T, we get (ST)^T = T^T S^T = TS. So (ST)^T = ST exactly when TS = ST. For the given pair,

ST = \begin{bmatrix} 2 & 7 \\ 0 & 2 \end{bmatrix}, \qquad TS = \begin{bmatrix} 2 & 0 \\ 7 & 2 \end{bmatrix}.

For instance, entry (1, 2) of ST is 1(1) + 2(3) = 7. The products differ, so ST is not symmetric, and TS is its transpose, as predicted.

Boundaries

The transpose of a product reverses the order. A^T B^T may not exist, may exist at the wrong size (Example 3), and for square matrices generally differs from (AB)^T.
Only a square matrix can be symmetric, though A^T A and A A^T are symmetric for A of any size.
Reading \mathbf{u}^T \mathbf{v} as the number \mathbf{u} \cdot \mathbf{v} is a convention for 1 \times 1 matrices; \mathbf{u} \mathbf{v}^T is an n \times n matrix.
A product of symmetric matrices is symmetric only when they commute (Example 7).
All of this is for real entries. Complex matrices pair the transpose with complex conjugation, outside this lesson.

Where this goes next

The identity (A\mathbf{x}) \cdot \mathbf{y} = \mathbf{x} \cdot (A^T \mathbf{y}) ties the columns of A to the vectors perpendicular to them (Orthogonal vectors). Projecting onto the span of the columns of A (Orthogonal bases and projection onto a subspace) and solving A\mathbf{x} = \mathbf{b} as nearly as possible (Least squares: the best approximate solution of Ax = b) both run through the normal equations A^T A \mathbf{x} = A^T \mathbf{b}. A matrix and its transpose share a determinant (Properties of determinants: row operations, products, transposes) and a rank (Rank). Symmetric matrices go on to have a deep result of their own, building on Diagonalization: each is diagonalizable with perpendicular eigenvectors.

Exercises

Computing transposes

Find the transpose of \begin{bmatrix} 3 & -2 \\ 1 & 0 \\ 4 & 6 \end{bmatrix} and give its size.
Find the transpose of the row \begin{bmatrix} 0 & 8 & -3 & 1 \end{bmatrix}.
Let M = \begin{bmatrix} 1 & 4 & -2 \\ 0 & 3 & 7 \\ 5 & -6 & 9 \end{bmatrix}. Write M^T and name (M^T)_{23}.
For the matrix M of Exercise 3, compute M + M^T.
Let F = \begin{bmatrix} 2 & -1 \\ 4 & 0 \end{bmatrix} and G = \begin{bmatrix} 1 & 5 \\ -3 & 2 \end{bmatrix}. Compute (3F - G^T)^T.

Symmetric matrices

Which are symmetric? (a) \begin{bmatrix} 4 & -1 \\ -1 & 0 \end{bmatrix} (b) \begin{bmatrix} 1 & 2 \\ 3 & 1 \end{bmatrix} (c) \begin{bmatrix} 0 & 2 & -5 \\ 2 & 7 & 1 \\ -5 & 1 & 3 \end{bmatrix} (d) \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}
Find a, b and c so that \begin{bmatrix} 5 & a & 2b \\ 3 & -1 & c \\ 8 & b + 1 & 0 \end{bmatrix} is symmetric.
The club grows to 9 huts. How many trail distances fill its symmetric table, with 0 on the diagonal?
How many entries can be chosen freely in a symmetric 6 \times 6 matrix?

Products and transposes

Let P = \begin{bmatrix} 1 & 2 \\ -1 & 3 \end{bmatrix} and Q = \begin{bmatrix} 0 & 4 & 1 \\ 2 & -1 & 5 \end{bmatrix}. Compute (PQ)^T and Q^T P^T, and say why P^T Q^T is not defined.
Let \mathbf{u} = \begin{bmatrix} 1 \\ -3 \\ 2 \end{bmatrix} and \mathbf{v} = \begin{bmatrix} 5 \\ 1 \\ 4 \end{bmatrix}. Compute \mathbf{u}^T \mathbf{v} and \mathbf{u} \mathbf{v}^T.
For K = \begin{bmatrix} 2 & 0 & -1 \\ 1 & 3 & 1 \end{bmatrix}, compute K^T K and K K^T.
For A = \begin{bmatrix} 1 & 2 \\ 0 & -1 \\ 3 & 1 \end{bmatrix}, \mathbf{x} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} and \mathbf{y} = \begin{bmatrix} 1 \\ 4 \\ -1 \end{bmatrix}, compute (A\mathbf{x}) \cdot \mathbf{y} and \mathbf{x} \cdot (A^T \mathbf{y}).
For A = \begin{bmatrix} 2 & 1 \\ 5 & 3 \end{bmatrix}, compute (A^{-1})^T and (A^T)^{-1} (Inverse of a 2 by 2 matrix, and solving with it).
Show that A^T A and A A^T differ for the square matrix A = \begin{bmatrix} 1 & 1 \\ 0 & 0 \end{bmatrix}.

In context

Loeiz copied a table of trail distances between three huts as \begin{bmatrix} 0 & 4 & 9 \\ 4 & 0 & 7 \\ 9 & 5 & 0 \end{bmatrix}. Which entries cannot both be right?
Three huts stand at heights of 1200, 1650 and 2100 meters. Let E have entry e_{ij} = h_j - h_i, the net climb in meters from hut i to hut j. Write E and show that E^T = -E.

Reasoning

Prove that (ABC)^T = C^T B^T A^T whenever ABC is defined.
Let S be a symmetric n \times n matrix and P any n \times k matrix. Prove that P^T S P is symmetric.
Prove that the diagonal entries of A^T A are the squared lengths of the columns of A, and deduce that A^T A = 0 forces A = 0.
A square matrix K is skew-symmetric when K^T = -K. Prove that every diagonal entry of a skew-symmetric matrix is 0.
Prove that every square matrix A is a symmetric matrix plus a skew-symmetric one, using \tfrac{1}{2}(A + A^T) and \tfrac{1}{2}(A - A^T). Then split \begin{bmatrix} 1 & 4 \\ -2 & 3 \end{bmatrix} this way.
True or false: if S is symmetric and invertible, then S^{-1} is symmetric. Give a reason.

Answers

\begin{bmatrix} 3 & 1 & 4 \\ -2 & 0 & 6 \end{bmatrix}, size 2 \times 3.
The column \begin{bmatrix} 0 \\ 8 \\ -3 \\ 1 \end{bmatrix}.
M^T = \begin{bmatrix} 1 & 0 & 5 \\ 4 & 3 & -6 \\ -2 & 7 & 9 \end{bmatrix}, and (M^T)_{23} = m_{32} = -6.
\begin{bmatrix} 2 & 4 & 3 \\ 4 & 6 & 1 \\ 3 & 1 & 18 \end{bmatrix}, symmetric by Result 5.
\begin{bmatrix} 5 & 7 \\ 0 & -2 \end{bmatrix}. By Results 2 and 1, (3F - G^T)^T = 3F^T - G, with 3F^T = \begin{bmatrix} 6 & 12 \\ -3 & 0 \end{bmatrix}.
(a) and (c) are symmetric. In (b) the mirror entries 2 and 3 differ; (d) is not square.
a = 3, b = 4, c = 5, from a = 3, 2b = 8 and c = b + 1.
36 distances, \tfrac{9 \cdot 8}{2} mirror pairs.
21 entries: 6 on the diagonal and 15 above it; the rest are copies.
(PQ)^T = Q^T P^T = \begin{bmatrix} 4 & 6 \\ 2 & -7 \\ 11 & 14 \end{bmatrix}, from PQ = \begin{bmatrix} 4 & 2 & 11 \\ 6 & -7 & 14 \end{bmatrix}. P^T is 2 \times 2 and Q^T is 3 \times 2; 2 \ne 3, so P^T Q^T does not exist.
\mathbf{u}^T \mathbf{v} = 5 - 3 + 8 = 10, and \mathbf{u} \mathbf{v}^T = \begin{bmatrix} 5 & 1 & 4 \\ -15 & -3 & -12 \\ 10 & 2 & 8 \end{bmatrix}.
K^T K = \begin{bmatrix} 5 & 3 & -1 \\ 3 & 9 & 3 \\ -1 & 3 & 2 \end{bmatrix} and K K^T = \begin{bmatrix} 5 & 1 \\ 1 & 11 \end{bmatrix}.
Both are -7: A\mathbf{x} = (4, -1, 7) gives 4 - 4 - 7, and A^T \mathbf{y} = (-2, -3) gives -4 - 3.
Both are \begin{bmatrix} 3 & -5 \\ -1 & 2 \end{bmatrix}. The determinant is 1 and A^{-1} = \begin{bmatrix} 3 & -1 \\ -5 & 2 \end{bmatrix}.
A^T A = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} but A A^T = \begin{bmatrix} 2 & 0 \\ 0 & 0 \end{bmatrix}.
Entries (2,3) and (3,2), reading 7 and 5; every other mirror pair agrees.
E = \begin{bmatrix} 0 & 450 & 900 \\ -450 & 0 & 450 \\ -900 & -450 & 0 \end{bmatrix}. Entry (i, j) of E^T is e_{ji} = h_i - h_j = -e_{ij}.
Result 3 for (AB) times C gives C^T (AB)^T, and again for AB gives C^T B^T A^T.
(P^T S P)^T = P^T S^T (P^T)^T = P^T S P, by Exercise 18, S^T = S and Result 1.
Entry (j, j) of A^T A is column j dotted with itself, \lVert \mathbf{a}_j \rVert^2. If A^T A = 0, every column has length 0, so is \mathbf{0}.
Entry (i, i) of K^T = -K reads k_{ii} = -k_{ii}, so k_{ii} = 0.
The parts add to A; the first is symmetric by Result 5, and (A - A^T)^T = A^T - A makes the second skew-symmetric. Here \begin{bmatrix} 1 & 1 \\ 1 & 3 \end{bmatrix} + \begin{bmatrix} 0 & 3 \\ -3 & 0 \end{bmatrix}.
True. By Result 6, (S^{-1})^T = (S^T)^{-1} = S^{-1}.