math.wiki
Linear algebra › Inner product spaces › Inner products and norms
Sign in

Inner products and norms

The dot product gave length, angle and the Cauchy-Schwarz inequality in R^n. Which of its properties did those proofs really use, and what do we get in polynomial spaces, function spaces and complex spaces once we keep only those?

Where you are 5 in Inner product spaces

Why this exists

Dot product, length and orthogonality in n dimensions proves the Cauchy-Schwarz and triangle inequalities from symmetry, linearity, and \mathbf{v} \cdot \mathbf{v} > 0 for \mathbf{v} \neq \mathbf{0}, never looking at an entry.

Other vector spaces need length and angle too. Amara and Mateo restore harpsichords. On their bench a string of length 1 is plucked at its middle; its shape is a continuous function on [0, 1], and the string also vibrates in the shapes \sin \pi x, \sin 2\pi x, and so on. How much of the plucked shape is the first of these? That is a question about length and angle in C[0, 1], where there are no entries to multiply.

Complex spaces add a problem: (1, i) \cdot (1, i) = 1 + i^2 = 0, a nonzero vector of length 0. An inner product keeps exactly the properties the proofs needed, adjusted for \mathbb{C}.

Definition

Throughout, \mathbb{F} is \mathbb{R} or \mathbb{C} (Complex numbers); for real c, \bar{c} = c.

An inner product on a vector space V over \mathbb{F} assigns to each pair \mathbf{u}, \mathbf{v} \in V a scalar \langle \mathbf{u}, \mathbf{v} \rangle \in \mathbb{F} such that, for all \mathbf{u}, \mathbf{v}, \mathbf{w} \in V and \lambda \in \mathbb{F}:

PropertyStatement
positivity\langle \mathbf{v}, \mathbf{v} \rangle is a real number and \langle \mathbf{v}, \mathbf{v} \rangle \ge 0
definiteness\langle \mathbf{v}, \mathbf{v} \rangle = 0 only when \mathbf{v} = \mathbf{0}
additivity in the first slot\langle \mathbf{u} + \mathbf{w}, \mathbf{v} \rangle = \langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{w}, \mathbf{v} \rangle
homogeneity in the first slot\langle \lambda \mathbf{u}, \mathbf{v} \rangle = \lambda \langle \mathbf{u}, \mathbf{v} \rangle
conjugate symmetry\langle \mathbf{u}, \mathbf{v} \rangle = \overline{\langle \mathbf{v}, \mathbf{u} \rangle}

Over \mathbb{R}, conjugate symmetry is plain symmetry. An inner product space is a vector space with one chosen inner product; it is real or complex with its scalars.

The norm of \mathbf{v} is \displaystyle \lVert \mathbf{v} \rVert = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}. Vectors are orthogonal, \mathbf{u} \perp \mathbf{v}, when \langle \mathbf{u}, \mathbf{v} \rangle = 0; by conjugate symmetry the order does not matter.

The two standard examples, used unless a lesson says otherwise:

  • the Euclidean inner product on \mathbb{F}^n, \langle \mathbf{u}, \mathbf{v} \rangle = u_1\bar{v}_1 + \dots + u_n\bar{v}_n, the dot product when \mathbb{F} = \mathbb{R};
  • the integral inner product on C[a, b], a < b: \langle f, g \rangle = \int_a^b f(x)g(x)\,dx (The definite integral).

Intuition

Each property is there because a proof needs it. Positivity and definiteness make the norm a real number that is 0 only for \mathbf{0}. Linearity in the first slot lets us expand. Conjugate symmetry is what complex scalars force: were the product linear in both slots, \langle i\mathbf{v}, i\mathbf{v} \rangle = i^2\langle \mathbf{v}, \mathbf{v} \rangle would be negative. With the conjugate, a scalar leaves the second slot as \bar{\lambda}, and \langle i\mathbf{v}, i\mathbf{v} \rangle = i\,\bar{i}\,\langle \mathbf{v}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{v} \rangle.

An inner product is also a choice: the plane carries many, each with its own unit circle and right angles.

\mathbf{u}
\mathbf{v}
\langle \mathbf{x}, \mathbf{x} \rangle = 1
\mathbf{x} \cdot \mathbf{x} = 1
Figure 1The dashed circle is \mathbf{x} \cdot \mathbf{x} = 1; the ellipse is \langle \mathbf{x}, \mathbf{x} \rangle = 1 for the inner product of Example 1. For that inner product \mathbf{u} = (1, 0) and \mathbf{v} = (-1, 2) are orthogonal and both have norm \displaystyle \sqrt{2}.

Results

Throughout, V is an inner product space over \mathbb{F}, with \mathbf{u}, \mathbf{v}, \mathbf{w} \in V and \lambda \in \mathbb{F}.

Result 1 (the standard examples). The Euclidean inner product is an inner product on \mathbb{F}^n, and the integral inner product is one on C[a, b].

Result 2 (the second slot). For fixed \mathbf{u}, the map \mathbf{v} \mapsto \langle \mathbf{v}, \mathbf{u} \rangle is a linear map V \to \mathbb{F}. Also \langle \mathbf{0}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{0} \rangle = 0, \langle \mathbf{u}, \mathbf{v} + \mathbf{w} \rangle = \langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{u}, \mathbf{w} \rangle and \langle \mathbf{u}, \lambda\mathbf{v} \rangle = \bar{\lambda}\langle \mathbf{u}, \mathbf{v} \rangle.

Result 3 (the norm). \lVert \mathbf{v} \rVert = 0 if and only if \mathbf{v} = \mathbf{0}; \lVert \lambda\mathbf{v} \rVert = |\lambda| \, \lVert \mathbf{v} \rVert; and

\lVert \mathbf{u} + \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2 + 2\operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle.

Result 4 (Pythagorean theorem). If \mathbf{u} \perp \mathbf{v}, then \lVert \mathbf{u} + \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2.

Result 5 (orthogonal decomposition). If \mathbf{v} \neq \mathbf{0}, exactly one scalar c makes (\mathbf{u} - c\mathbf{v}) \perp \mathbf{v}, namely c = \langle \mathbf{u}, \mathbf{v} \rangle / \lVert \mathbf{v} \rVert^2. So \mathbf{u} = c\mathbf{v} + \mathbf{w} with \mathbf{w} \perp \mathbf{v}.

Result 6 (Cauchy-Schwarz inequality). |\langle \mathbf{u}, \mathbf{v} \rangle| \le \lVert \mathbf{u} \rVert \, \lVert \mathbf{v} \rVert, with equality if and only if one of \mathbf{u}, \mathbf{v} is a scalar multiple of the other.

The angle. In a real inner product space, for nonzero \mathbf{u} and \mathbf{v}, Result 6 puts \displaystyle \frac{\langle \mathbf{u}, \mathbf{v} \rangle}{\lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert} in [-1, 1]; the angle between them is the \theta \in [0, \pi] with that cosine.

Result 7 (triangle inequality). \lVert \mathbf{u} + \mathbf{v} \rVert \le \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert, with equality if and only if one of \mathbf{u}, \mathbf{v} is a nonnegative real multiple of the other.

Result 8 (parallelogram law). \lVert \mathbf{u} + \mathbf{v} \rVert^2 + \lVert \mathbf{u} - \mathbf{v} \rVert^2 = 2\lVert \mathbf{u} \rVert^2 + 2\lVert \mathbf{v} \rVert^2.

Why it is true

Result 1. On \mathbb{F}^n, additivity and homogeneity in the first slot hold term by term, conjugate symmetry holds because \overline{v_k\bar{u}_k} = u_k\bar{v}_k, and \langle \mathbf{u}, \mathbf{u} \rangle = |u_1|^2 + \dots + |u_n|^2 is a sum of nonnegative reals, 0 only when every u_k = 0.

On C[a, b], fg is continuous, so the integral exists; symmetry holds since fg = gf, linearity is linearity of the integral, and positivity holds since f^2 \ge 0. For definiteness, let f(x_0) \neq 0. By Continuity of f^2 there is \delta > 0 with \displaystyle f(x)^2 > \frac{1}{2}f(x_0)^2 for every x \in [a, b] within \delta of x_0, and those x include an interval of length \ell = \min(\delta, b - a) > 0. As f^2 \ge 0 everywhere, \displaystyle \int_a^b f^2 \ge \frac{\ell}{2}f(x_0)^2 > 0.

Result 2. Additivity and homogeneity in the first slot say exactly that \mathbf{v} \mapsto \langle \mathbf{v}, \mathbf{u} \rangle is linear, so it sends \mathbf{0} to 0; then \langle \mathbf{v}, \mathbf{0} \rangle = \overline{\langle \mathbf{0}, \mathbf{v} \rangle} = 0. By conjugate symmetry, additivity, and conjugate symmetry again,

\begin{aligned} \langle \mathbf{u}, \mathbf{v} + \mathbf{w} \rangle &= \overline{\langle \mathbf{v}, \mathbf{u} \rangle + \langle \mathbf{w}, \mathbf{u} \rangle} \\ &= \langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{u}, \mathbf{w} \rangle, \end{aligned}

and likewise \langle \mathbf{u}, \lambda\mathbf{v} \rangle = \overline{\lambda\langle \mathbf{v}, \mathbf{u} \rangle} = \bar{\lambda}\langle \mathbf{u}, \mathbf{v} \rangle.

Result 3. The first claim is definiteness. Homogeneity and Result 2 give \langle \lambda\mathbf{v}, \lambda\mathbf{v} \rangle = \lambda\bar{\lambda}\langle \mathbf{v}, \mathbf{v} \rangle = |\lambda|^2\lVert \mathbf{v} \rVert^2. Expanding with additivity in both slots,

\begin{aligned} \lVert \mathbf{u} + \mathbf{v} \rVert^2 &= \langle \mathbf{u}, \mathbf{u} \rangle + \langle \mathbf{u}, \mathbf{v} \rangle \\ &\quad + \langle \mathbf{v}, \mathbf{u} \rangle + \langle \mathbf{v}, \mathbf{v} \rangle, \end{aligned}

and the middle terms are z + \bar{z} = 2\operatorname{Re} z for z = \langle \mathbf{u}, \mathbf{v} \rangle.

Result 4. In Result 3, \operatorname{Re} 0 = 0.

Result 5. By linearity in the first slot, \langle \mathbf{u} - c\mathbf{v}, \mathbf{v} \rangle = \langle \mathbf{u}, \mathbf{v} \rangle - c\lVert \mathbf{v} \rVert^2, which is 0 exactly for c = \langle \mathbf{u}, \mathbf{v} \rangle / \lVert \mathbf{v} \rVert^2, a number since \lVert \mathbf{v} \rVert \neq 0.

Result 6. If \mathbf{v} = \mathbf{0}, both sides are 0 and \mathbf{v} = 0\mathbf{u}. Otherwise write \mathbf{u} = c\mathbf{v} + \mathbf{w} as in Result 5. Since \langle c\mathbf{v}, \mathbf{w} \rangle = c\,\overline{\langle \mathbf{w}, \mathbf{v} \rangle} = 0, Results 4 and 3 give

\begin{aligned} \lVert \mathbf{u} \rVert^2 &= |c|^2\lVert \mathbf{v} \rVert^2 + \lVert \mathbf{w} \rVert^2 \\ &\ge |c|^2\lVert \mathbf{v} \rVert^2 = \frac{|\langle \mathbf{u}, \mathbf{v} \rangle|^2}{\lVert \mathbf{v} \rVert^2}. \end{aligned}

Multiply by \lVert \mathbf{v} \rVert^2 and take square roots. Equality holds exactly when \mathbf{w} = \mathbf{0}, that is, \mathbf{u} = c\mathbf{v}, which by the uniqueness in Result 5 means \mathbf{u} is a multiple of \mathbf{v}. And if \mathbf{v} = a\mathbf{u} with \mathbf{v} \neq \mathbf{0}, then a \neq 0 and \mathbf{u} = a^{-1}\mathbf{v}, so “one is a multiple of the other” is the same condition.

Result 7. Since \operatorname{Re} z \le |z|, Results 3 and 6 give

\begin{aligned} \lVert \mathbf{u} + \mathbf{v} \rVert^2 &\le \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2 + 2|\langle \mathbf{u}, \mathbf{v} \rangle| \\ &\le \big(\lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert\big)^2, \end{aligned}

and both sides are nonnegative. Equality needs \operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle = |\langle \mathbf{u}, \mathbf{v} \rangle| = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert. For \mathbf{v} = \mathbf{0} that holds, with \mathbf{v} = 0\mathbf{u}. For \mathbf{v} \neq \mathbf{0}, Result 6 gives \mathbf{u} = c\mathbf{v}, and c\lVert \mathbf{v} \rVert^2 = \langle \mathbf{u}, \mathbf{v} \rangle must be a nonnegative real, so c \ge 0. Conversely, \mathbf{u} = c\mathbf{v} with c \ge 0 makes both sides (1 + c)\lVert \mathbf{v} \rVert.

Result 8. Result 3 gives \lVert \mathbf{u} \pm \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2 \pm 2\operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle; add.

Worked examples

Example 1

A new inner product on the plane

(a) Show that \langle \mathbf{x}, \mathbf{y} \rangle = 2x_1y_1 + x_1y_2 + x_2y_1 + x_2y_2 is an inner product on \mathbb{R}^2 for which \mathbf{u} = (1, 0) and \mathbf{v} = (-1, 2) are orthogonal. (b) Show that x_1y_1 + 3x_1y_2 + 3x_2y_1 + x_2y_2 is not an inner product on \mathbb{R}^2.

Solution

(a) The formula is symmetric in \mathbf{x} and \mathbf{y}, and each term has exactly one factor from \mathbf{x}, so it is linear in \mathbf{x}. Completing the square,

\langle \mathbf{x}, \mathbf{x} \rangle = 2x_1^2 + 2x_1x_2 + x_2^2 = x_1^2 + (x_1 + x_2)^2 \ge 0,

which is 0 only if x_1 = 0 and x_1 + x_2 = 0, that is, \mathbf{x} = \mathbf{0}. So it is an inner product. Now \langle \mathbf{u}, \mathbf{v} \rangle = -2 + 2 = 0, and \lVert \mathbf{u} \rVert^2 = \lVert \mathbf{v} \rVert^2 = 2. Yet \mathbf{u} \cdot \mathbf{v} = -1: the vectors are not perpendicular on the page.

(b) Positivity fails: at \mathbf{x} = (1, -1) the formula gives 1 - 3 - 3 + 1 = -4.

Example 2

Conjugates in \mathbb{C}^3

Let \mathbf{u} = (1 + i, 2, -i) and \mathbf{v} = (2, 1 - i, 3i). Find \langle \mathbf{u}, \mathbf{v} \rangle, \langle \mathbf{v}, \mathbf{u} \rangle, \langle i\mathbf{u}, \mathbf{v} \rangle, \langle \mathbf{u}, i\mathbf{v} \rangle and both norms, and confirm Cauchy-Schwarz.

Solution

The second vector is conjugated, \bar{\mathbf{v}} = (2, 1 + i, -3i):

\begin{aligned} \langle \mathbf{u}, \mathbf{v} \rangle &= (1 + i)2 + 2(1 + i) + (-i)(-3i) \\ &= 4 + 4i - 3 = 1 + 4i. \end{aligned}

By conjugate symmetry \langle \mathbf{v}, \mathbf{u} \rangle = 1 - 4i. By Result 2, \langle i\mathbf{u}, \mathbf{v} \rangle = i(1 + 4i) = -4 + i, and \langle \mathbf{u}, i\mathbf{v} \rangle = -i(1 + 4i) = 4 - i.

Norms squared are sums of squared moduli (Modulus, argument and polar form): \lVert \mathbf{u} \rVert^2 = 2 + 4 + 1 = 7 and \lVert \mathbf{v} \rVert^2 = 4 + 2 + 9 = 15. Cauchy-Schwarz holds: \displaystyle |1 + 4i| = \sqrt{17} \approx 4.12 \le \sqrt{105} \approx 10.25.

Example 3

An inner product on polynomials by evaluation

Prove that \langle p, q \rangle = p(-1)q(-1) + p(0)q(0) + p(1)q(1) is an inner product on \mathcal{P}_2(\mathbb{R}), and find \lVert 1 \rVert, \lVert x \rVert, \lVert x^2 \rVert, \langle 1, x^2 \rangle and \langle x, x^2 \rangle. Show it is not an inner product on \mathcal{P}_3(\mathbb{R}).

Solution

Symmetry is clear, and the formula is linear in p because evaluation is: (p + r)(t) = p(t) + r(t) and (\lambda p)(t) = \lambda p(t). Next, \langle p, p \rangle = p(-1)^2 + p(0)^2 + p(1)^2 \ge 0, and if it is 0, then p has the roots -1, 0, 1. A nonzero polynomial of degree at most 2 has at most two roots (The remainder and factor theorems), so p = 0.

The values of 1, x, x^2 at -1, 0, 1 are (1, 1, 1), (-1, 0, 1), (1, 0, 1). So \displaystyle \lVert 1 \rVert = \sqrt{3}, \displaystyle \lVert x \rVert = \lVert x^2 \rVert = \sqrt{2}, \langle 1, x^2 \rangle = 2 and \langle x, x^2 \rangle = -1 + 0 + 1 = 0.

On \mathcal{P}_3(\mathbb{R}) definiteness fails: x^3 - x is not the zero polynomial but vanishes at -1, 0, 1.

Example 4

How much of a plucked string is the first mode

The plucked test string has the shape \displaystyle f(x) = \frac{1}{2} - \left|x - \frac{1}{2}\right| on [0, 1]. In C[0, 1], find the angle between f and s(x) = \sin \pi x, write f = cs + w with w \perp s, and find \lVert w \rVert.

Solution

Both f and s are unchanged when x is replaced by 1 - x, so each integral over [0, 1] is twice the one over \displaystyle \left[0, \frac{1}{2}\right], where f(x) = x. By parts,

\begin{aligned} \int_0^{1/2} x\sin \pi x\,dx &= \Big[-\tfrac{x\cos \pi x}{\pi}\Big]_0^{1/2} \\ &\quad + \frac{1}{\pi}\int_0^{1/2}\cos \pi x\,dx = \frac{1}{\pi^2}, \end{aligned}

so \displaystyle \langle f, s \rangle = \frac{2}{\pi^2}. Also \displaystyle \lVert f \rVert^2 = 2\int_0^{1/2} x^2\,dx = \frac{1}{12} and \displaystyle \lVert s \rVert^2 = \int_0^1 \sin^2 \pi x\,dx = \frac{1}{2}. Then

\cos\theta = \frac{2/\pi^2}{\sqrt{1/12}\,\sqrt{1/2}} = \frac{4\sqrt{6}}{\pi^2} \approx 0.9927,

so \theta \approx 6.9^\circ: the shape is nearly the first mode.

By Result 5, \displaystyle c = \frac{\langle f, s \rangle}{\lVert s \rVert^2} = \frac{4}{\pi^2} \approx 0.405. Since cs \perp w, the Pythagorean theorem gives

\lVert w \rVert^2 = \lVert f \rVert^2 - c^2\lVert s \rVert^2 = \frac{1}{12} - \frac{8}{\pi^4} \approx 0.00121,

so \lVert w \rVert \approx 0.035, about 12\% of \lVert f \rVert \approx 0.289.

0 1/2 1
f
\tfrac{4}{\pi^2}\sin \pi x
Figure 2The plucked shape f and its component \displaystyle \frac{4}{\pi^2}\sin \pi x along the first mode. The difference is orthogonal to \sin \pi x and has about 12\% of the norm of f.
Example 5

A norm that comes from no inner product

On C[0, 1] let \lVert f \rVert_{\max} = \max_{0 \le x \le 1} |f(x)|, which exists by the The extreme value theorem. Checked pointwise, it is a norm in the general sense: nonnegative, 0 only for 0, \lVert \lambda f \rVert = |\lambda| \lVert f \rVert, and the triangle inequality, like every inner product norm (Results 3 and 7). Prove that no inner product on C[0, 1] has \lVert f \rVert_{\max} as its norm.

Solution

By Result 8, a norm that comes from an inner product satisfies the parallelogram law for every pair, so one failing pair is enough. Take f(x) = x and g(x) = 1 - x. Then f + g = 1 and f - g = 2x - 1, so

\lVert f + g \rVert_{\max}^2 + \lVert f - g \rVert_{\max}^2 = 1 + 1 = 2,

while 2\lVert f \rVert_{\max}^2 + 2\lVert g \rVert_{\max}^2 = 2 + 2 = 4. The integral norm passes, as it must: \displaystyle 1 + \frac{1}{3} = \frac{4}{3} = 2 \cdot \frac{1}{3} + 2 \cdot \frac{1}{3}.

Example 6

Cauchy-Schwarz as an integral inequality

Prove that every f \in C[0, 1] satisfies

\left|\int_0^1 x f(x)\,dx\right| \le \frac{1}{\sqrt{3}}\left(\int_0^1 f(x)^2\,dx\right)^{1/2},

with equality exactly when f(x) = kx for a real k. Then test f(x) = e^x.

Solution

With h(x) = x, the left side is |\langle f, h \rangle| and \displaystyle \lVert h \rVert^2 = \int_0^1 x^2\,dx = \frac{1}{3}, so the claim is |\langle f, h \rangle| \le \lVert h \rVert \lVert f \rVert, Result 6. Equality holds exactly when one of f, h is a multiple of the other, which, as h \neq 0, means f = kh.

For e^x, \int_0^1 xe^x\,dx = \big[(x - 1)e^x\big]_0^1 = 1 and \displaystyle \int_0^1 e^{2x}\,dx = \frac{e^2 - 1}{2}, so the right side is \displaystyle \sqrt{\frac{e^2 - 1}{6}} \approx 1.032, above 1.

Boundaries

  • Over \mathbb{C} the Pythagorean theorem has no converse: \mathbf{u} = (1, i) and \mathbf{v} = (i, 0) give \lVert \mathbf{u} + \mathbf{v} \rVert^2 = 3 = 2 + 1 although \langle \mathbf{u}, \mathbf{v} \rangle = -i; Result 3 only forces \operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle = 0. So an angle is defined only in real spaces.
  • Definiteness is the property most easily lost. On all integrable functions on [0, 1], the function that is 1 at \displaystyle x = \frac{1}{2} and 0 elsewhere has \int_0^1 f^2 = 0, which is why the integral inner product lives on continuous functions.
  • A norm that fails the parallelogram law, such as \lVert f \rVert_{\max}, comes from no inner product. Conversely, a norm satisfying the law does come from one (the Jordan-von Neumann theorem); that proof is longer and not given here.

Where this goes next

Orthonormal bases and Gram-Schmidt, proved applies Result 5 over and over in Gram-Schmidt, after which every coordinate is an inner product. Inner products then define the adjoint of an operator, and self-adjoint operators lead to the The spectral theorem. In Fourier series, \sin \pi x, \sin 2\pi x, \dots are orthogonal, and the coefficient of Example 4 is the first of many.

Exercises

Checking the axioms

  1. Is \langle \mathbf{x}, \mathbf{y} \rangle = 3x_1y_1 - x_1y_2 - x_2y_1 + x_2y_2 an inner product on \mathbb{R}^2?
  2. Is \langle \mathbf{x}, \mathbf{y} \rangle = x_1y_1 + x_1y_2 + x_2y_1 + x_2y_2 an inner product on \mathbb{R}^2? If not, which property fails?
  3. Is \langle \mathbf{z}, \mathbf{w} \rangle = z_1\bar{w}_1 + iz_2\bar{w}_2 an inner product on \mathbb{C}^2? If not, which properties fail?
  4. (a) Is \langle p, q \rangle = p(0)q(0) + p(1)q(1) an inner product on \mathcal{P}_1(\mathbb{R})? (b) On \mathcal{P}_2(\mathbb{R})?
  5. On \mathbb{R}^{2 \times 2}, \langle A, B \rangle = \operatorname{tr}(B^{T}A) is the dot product of the four entries. For A = \begin{bmatrix} 1 & 2 \\ 0 & -1 \end{bmatrix} and B = \begin{bmatrix} 3 & -1 \\ 2 & 4 \end{bmatrix}, find \langle A, B \rangle, \lVert A \rVert and \lVert B \rVert, and decide whether A \perp I_2.

Computing in inner product spaces

In Exercises 6 and 7, \mathbf{u} = (2 - i, 1 + 2i) and \mathbf{v} = (1 + i, -i) in \mathbb{C}^2.

  1. Find \langle \mathbf{u}, \mathbf{v} \rangle, \langle \mathbf{v}, \mathbf{u} \rangle, \lVert \mathbf{u} \rVert, \lVert \mathbf{v} \rVert and \langle (1 + i)\mathbf{u}, \mathbf{v} \rangle, and confirm Cauchy-Schwarz.
  2. Find the scalar c with (\mathbf{u} - c\mathbf{v}) \perp \mathbf{v}, and find \lVert \mathbf{u} - c\mathbf{v} \rVert^2.
  3. In C[-1, 1], find \langle x, 1 - x \rangle, both norms, and the angle between x and 1 - x.
  4. In C[-\pi, \pi], find \langle x, \sin x \rangle, \lVert x \rVert, \lVert \sin x \rVert, and the angle between x and \sin x to the nearest tenth of a degree.
  5. For the plucked shape f of Example 4, find \langle f, \sin 2\pi x \rangle and \langle f, \sin 3\pi x \rangle in C[0, 1].
  6. On \mathbb{R}^3 take \langle \mathbf{x}, \mathbf{y} \rangle = x_1y_1 + 2x_2y_2 + 5x_3y_3. For \mathbf{a} = (1, 1, 1) and \mathbf{b} = (2, -1, 0), find \langle \mathbf{a}, \mathbf{b} \rangle, \mathbf{a} \cdot \mathbf{b}, \lVert \mathbf{a} \rVert, \lVert \mathbf{b} \rVert and \lVert \mathbf{a} + \mathbf{b} \rVert^2, and say which theorem the last illustrates.

Norms and the parallelogram law

  1. In a real inner product space, \lVert \mathbf{u} \rVert = 2, \lVert \mathbf{v} \rVert = 5 and \lVert \mathbf{u} - \mathbf{v} \rVert = 4. Find \lVert \mathbf{u} + \mathbf{v} \rVert and \langle \mathbf{u}, \mathbf{v} \rangle.
  2. In a complex inner product space, \lVert \mathbf{u} \rVert = 1, \lVert \mathbf{v} \rVert = 2 and \lVert \mathbf{u} + \mathbf{v} \rVert = 3. Find \lVert \mathbf{u} - \mathbf{v} \rVert and \langle \mathbf{u}, \mathbf{v} \rangle, and write \mathbf{v} in terms of \mathbf{u}.
  3. On \mathcal{P}_1(\mathbb{R}), \lVert p \rVert = |p(0)| + |p(1)| is a norm. Does it come from an inner product?

Proofs

  1. Prove that \langle \mathbf{x}, \mathbf{y} \rangle = ax_1y_1 + b(x_1y_2 + x_2y_1) + dx_2y_2, with a, b, d real, is an inner product on \mathbb{R}^2 if and only if a > 0 and ad - b^2 > 0.
  2. Let c_1, \dots, c_n be real. Prove that \langle \mathbf{x}, \mathbf{y} \rangle = c_1x_1\bar{y}_1 + \dots + c_nx_n\bar{y}_n is an inner product on \mathbb{F}^n if and only if every c_k > 0.
  3. Prove that \mathbf{u} \perp \mathbf{v} if and only if \lVert \mathbf{u} + a\mathbf{v} \rVert = \lVert \mathbf{u} - a\mathbf{v} \rVert for every a \in \mathbb{F}.
  4. Prove that in a real inner product space, \lVert \mathbf{u} + \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2 implies \mathbf{u} \perp \mathbf{v}. Give a pair in \mathbb{C}^2, other than the one in Boundaries, where this fails.
  5. Prove that two inner products on the same vector space that give the same norm are equal.
  6. Let t_0, \dots, t_m be distinct reals, m \ge 1. Prove that \langle p, q \rangle = p(t_0)q(t_0) + \dots + p(t_m)q(t_m) is an inner product on \mathcal{P}_m(\mathbb{R}), and that the sum over only t_1, \dots, t_m is not.
Answers
  1. Yes. It is symmetric and linear in \mathbf{x}, and \langle \mathbf{x}, \mathbf{x} \rangle = 2x_1^2 + (x_1 - x_2)^2 \ge 0, which is 0 only when x_1 = 0 and x_2 = x_1 = 0.
  2. No. \langle \mathbf{x}, \mathbf{x} \rangle = (x_1 + x_2)^2 \ge 0, but definiteness fails: \mathbf{x} = (1, -1) gives 0.
  3. No. \langle \mathbf{e}_2, \mathbf{e}_2 \rangle = i is not a nonnegative real, so positivity fails, and so does conjugate symmetry, since \overline{i} \neq i.
  4. (a) Yes. It is symmetric, linear in p, and p(0)^2 + p(1)^2 = 0 gives a polynomial of degree at most 1 two roots, so p = 0. (b) No: x^2 - x is nonzero with \langle p, p \rangle = 0.
  5. \langle A, B \rangle = 3 - 2 + 0 - 4 = -3, \displaystyle \lVert A \rVert = \sqrt{6}, \displaystyle \lVert B \rVert = \sqrt{30}. Yes, A \perp I_2, since \langle A, I_2 \rangle = \operatorname{tr} A = 0.
  6. \langle \mathbf{u}, \mathbf{v} \rangle = (2 - i)(1 - i) + (1 + 2i)(i) = (1 - 3i) + (-2 + i) = -1 - 2i and \langle \mathbf{v}, \mathbf{u} \rangle = -1 + 2i. \displaystyle \lVert \mathbf{u} \rVert = \sqrt{10}, \displaystyle \lVert \mathbf{v} \rVert = \sqrt{3}, and \langle (1 + i)\mathbf{u}, \mathbf{v} \rangle = (1 + i)(-1 - 2i) = 1 - 3i. Cauchy-Schwarz: \displaystyle \sqrt{5} \le \sqrt{30}.
  7. \displaystyle c = \frac{\langle \mathbf{u}, \mathbf{v} \rangle}{\lVert \mathbf{v} \rVert^2} = \frac{-1 - 2i}{3} by Result 5. By the Pythagorean theorem, \displaystyle \lVert \mathbf{u} - c\mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 - |c|^2\lVert \mathbf{v} \rVert^2 = 10 - \frac{5}{9} \cdot 3 = \frac{25}{3}.
  8. \displaystyle \langle x, 1 - x \rangle = \int_{-1}^{1} (x - x^2)\,dx = -\frac{2}{3}, \displaystyle \lVert x \rVert = \sqrt{2/3} and \displaystyle \lVert 1 - x \rVert = \sqrt{8/3}, so \displaystyle \cos\theta = -\frac{2/3}{4/3} = -\frac{1}{2} and \displaystyle \theta = \frac{2\pi}{3}.
  9. \langle x, \sin x \rangle = 2\pi, \displaystyle \lVert x \rVert = \sqrt{2\pi^3/3}, \displaystyle \lVert \sin x \rVert = \sqrt{\pi}. So \displaystyle \cos\theta = \frac{\sqrt{6}}{\pi} \approx 0.7797 and \theta \approx 38.8^\circ.
  10. \langle f, \sin 2\pi x \rangle = 0: under x \mapsto 1 - x, f is unchanged and \sin 2\pi x changes sign, so the two halves cancel. \sin 3\pi x is unchanged, so \displaystyle \langle f, \sin 3\pi x \rangle = 2\int_0^{1/2} x\sin 3\pi x\,dx = -\frac{2}{9\pi^2}, by parts as in Example 4.
  11. \langle \mathbf{a}, \mathbf{b} \rangle = 2 - 2 + 0 = 0 while \mathbf{a} \cdot \mathbf{b} = 1. \displaystyle \lVert \mathbf{a} \rVert = \sqrt{8}, \displaystyle \lVert \mathbf{b} \rVert = \sqrt{6}, and \mathbf{a} + \mathbf{b} = (3, 0, 1) gives 9 + 0 + 5 = 14 = 8 + 6, the Pythagorean theorem for this inner product.
  12. By the parallelogram law \lVert \mathbf{u} + \mathbf{v} \rVert^2 = 2(4 + 25) - 16 = 42, so \displaystyle \lVert \mathbf{u} + \mathbf{v} \rVert = \sqrt{42}. From 4 + 25 - 2\langle \mathbf{u}, \mathbf{v} \rangle = 16, \displaystyle \langle \mathbf{u}, \mathbf{v} \rangle = \frac{13}{2}.
  13. \lVert \mathbf{u} - \mathbf{v} \rVert^2 = 2(1 + 4) - 9 = 1, so \lVert \mathbf{u} - \mathbf{v} \rVert = 1. Result 3 gives \operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle = 2, and |\langle \mathbf{u}, \mathbf{v} \rangle| \le 2, so \langle \mathbf{u}, \mathbf{v} \rangle = 2. Equality in Result 7 gives \mathbf{v} = c\mathbf{u} with c \ge 0, and \lVert \mathbf{v} \rVert = 2 forces \mathbf{v} = 2\mathbf{u}.
  14. No. For p = 1, q = 2x - 1: \lVert p \rVert = \lVert q \rVert = 2, \lVert p + q \rVert = \lVert 2x \rVert = 2 and \lVert p - q \rVert = \lVert 2 - 2x \rVert = 2, so 4 + 4 = 8 \neq 16 = 2 \cdot 4 + 2 \cdot 4 and the parallelogram law fails.
  15. Symmetry and linearity in \mathbf{x} hold for all a, b, d, and \langle \mathbf{x}, \mathbf{x} \rangle = ax_1^2 + 2bx_1x_2 + dx_2^2. If a > 0 and ad - b^2 > 0, completing the square gives \langle \mathbf{x}, \mathbf{x} \rangle = a\big(x_1 + \tfrac{b}{a}x_2\big)^2 + \tfrac{ad - b^2}{a}x_2^2 \ge 0, which is 0 only if x_2 = 0 and then x_1 = 0. Conversely, for an inner product, a = \langle \mathbf{e}_1, \mathbf{e}_1 \rangle > 0, and \mathbf{x} = (-b, a) \neq \mathbf{0} gives 0 < ab^2 - 2ab^2 + da^2 = a(ad - b^2), so ad - b^2 > 0.
  16. Linearity in \mathbf{x} holds term by term, and as each c_k is real, \overline{c_ky_k\bar{x}_k} = c_kx_k\bar{y}_k, so conjugate symmetry holds. If every c_k > 0, then \langle \mathbf{x}, \mathbf{x} \rangle = \sum c_k|x_k|^2 \ge 0, with equality only if every x_k = 0. Conversely, for an inner product, c_k = \langle \mathbf{e}_k, \mathbf{e}_k \rangle > 0 since \mathbf{e}_k \neq \mathbf{0}.
  17. Norms are nonnegative, so compare squares. By Results 3 and 2, \lVert \mathbf{u} + a\mathbf{v} \rVert^2 - \lVert \mathbf{u} - a\mathbf{v} \rVert^2 = 4\operatorname{Re}\big(\bar{a}\langle \mathbf{u}, \mathbf{v} \rangle\big). If \langle \mathbf{u}, \mathbf{v} \rangle = 0 this is 0 for every a. Conversely, a = 1 gives \operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle = 0, which settles the real case; over \mathbb{C}, a = i gives \operatorname{Re}\big(-i\langle \mathbf{u}, \mathbf{v} \rangle\big) = \operatorname{Im}\langle \mathbf{u}, \mathbf{v} \rangle = 0.
  18. Over \mathbb{R}, \operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle = \langle \mathbf{u}, \mathbf{v} \rangle, so by Result 3 the equation says 2\langle \mathbf{u}, \mathbf{v} \rangle = 0. Over \mathbb{C}, take \mathbf{u} = (1, 0), \mathbf{v} = (i, 0): \langle \mathbf{u}, \mathbf{v} \rangle = -i \neq 0, yet \lVert \mathbf{u} + \mathbf{v} \rVert^2 = |1 + i|^2 = 2 = 1 + 1.
  19. Result 3 for \mathbf{u} \pm \mathbf{v} gives \displaystyle \operatorname{Re}\langle \mathbf{u}, \mathbf{v} \rangle = \frac{1}{4}\big(\lVert \mathbf{u} + \mathbf{v} \rVert^2 - \lVert \mathbf{u} - \mathbf{v} \rVert^2\big), built from the norm alone, so two inner products with the same norm have the same real parts; over \mathbb{R} that finishes it. Over \mathbb{C}, Result 2 gives \langle \mathbf{u}, i\mathbf{v} \rangle = -i\langle \mathbf{u}, \mathbf{v} \rangle, whose real part is \operatorname{Im}\langle \mathbf{u}, \mathbf{v} \rangle; so the imaginary parts agree too.
  20. As in Example 3: the formula is symmetric, linear in p because evaluation is linear, and \langle p, p \rangle = \sum p(t_k)^2 \ge 0. If it is 0, p has m + 1 distinct roots, too many for a nonzero polynomial of degree at most m (The remainder and factor theorems), so p = 0. With only t_1, \dots, t_m, the nonzero p = (x - t_1)\cdots(x - t_m) \in \mathcal{P}_m(\mathbb{R}) has \langle p, p \rangle = 0.
math.wiki · CC-BY-SA-4.0