What is inner product? — Mathematics

Definition 23.1 University Mathematics — Year 1 · Chapter 23 — Euclidean Spaces

An inner product on $E$ is a map $\langle\cdot,\cdot\rangle \colon E \times E \to \R$ that is bilinear, symmetric, and positive definite ( $\langle x, x\rangle > 0$ for $x \neq 0$ ). A finite-dimensional space so equipped is a Euclidean space. The norm of $x$ is $\norm{x} = \sqrt{\langle x, x\rangle}$ , and $d(x, y) = \norm{x - y}$ .

Examples

Example 23.2

On $\R^n$ : the canonical product $\langle x, y\rangle = \sum x_i y_i$ . On $C(\intcc{a}{b})$ : $\langle f, g \rangle = \int_a^b fg$ (positive definiteness is Theorem 15.7 (4)). On $\R_n[X]$ : $\langle P, Q\rangle = \int_0^1 PQ$ , or $\sum_{i} P(x_i)Q(x_i)$ over $n+1$ distinct points.

Example 23.3 (The angle between two polynomials)

Once an inner product is chosen, any two nonzero vectors have an angle, via $\cos\theta = \frac{\langle x, y\rangle}{\norm x\,\norm y}$ (a legitimate cosine by Cauchy–Schwarz). For $X$ and $X^2$ in $\int_0^1$ :

\langle X, X^2\rangle = \frac14, \qquad \norm X = \frac1{\sqrt3}, \qquad \norm{X^2} = \frac1{\sqrt5}, \qquad \cos\theta = \frac{1/4}{1/\sqrt{15}} = \frac{\sqrt{15}}{4} \approx 0.968 :

an angle of about $14.5$ degrees — on $\intcc{0}{1}$ , the graphs of $x$ and $x^2$ are “nearly parallel” in the quadratic-mean sense, which is why removing that shared direction (Gram–Schmidt, below) leaves only the small correction $X^2 - X + \frac16$ .

Example 23.7 (Orthonormal coordinates, with a Parseval check)

Expand $x = (1, 2, 3)$ in the orthonormal basis of Exercise 23.3,

e_1 = \tfrac{1}{\sqrt2}(1,1,0), \quad e_2 = \tfrac{1}{\sqrt6}(1,-1,2), \quad e_3 = \tfrac{1}{\sqrt3}(-1,1,1).

No system to solve — three inner products:

\langle x, e_1\rangle = \frac{3}{\sqrt2}, \qquad \langle x, e_2\rangle = \frac{1 - 2 + 6}{\sqrt6} = \frac{5}{\sqrt6}, \qquad \langle x, e_3\rangle = \frac{-1 + 2 + 3}{\sqrt3} = \frac{4}{\sqrt3}.

Certification by the norm formula of the proposition:

\frac{9}{2} + \frac{25}{6} + \frac{16}{3} = \frac{27 + 25 + 32}{6} = 14 = \norm{x}^2 = 1 + 4 + 9 .

This sum-of-squared-coordinates check (a finite Parseval identity) costs seconds and catches sign and normalization errors with near certainty — make it a habit whenever an orthonormal expansion is computed; its infinite-dimensional version, for the Fourier coefficients of Example 23.14, is a theorem of the Year 3 volume.

Read in context →

Definition 13.1 University Mathematics — Year 3 · Chapter 13 — Hilbert Spaces

An inner product is a map $\langle \cdot,\cdot\rangle \colon H\times H \to K$ , linear in the second variable, with $\langle y, x\rangle = \overline{\langle x, y\rangle}$ and $\langle x, x\rangle > 0$ for $x \neq 0$ . It induces the norm $\norm x = \langle x, x\rangle^{1/2}$ , the Cauchy–Schwarz inequality $\abs{\langle x, y\rangle} \leq \norm x\norm y$ (Year 2’s proof — the discriminant — is unchanged), and the parallelogram law

\norm{x + y}^2 + \norm{x - y}^2 = 2\norm x^2 + 2\norm y^2 .

A Hilbert space is an inner-product space complete for this norm. Examples: $\ell^2$ (Problem 8.1) and, the fundamental one, $L^2(\mu)$ with $\langle f, g\rangle = \int\bar fg\,\dd\mu$ — complete by Riesz–Fischer (Theorem 12.4); the inner product is finite by Cauchy–Schwarz ( $=$ Hölder at $p = q = 2$ ).

Examples

Example 13.5 (A projection, computed to the end)

In $H = L^2(\intcc01)$ , what is the best approximation of $f(x) = x^2$ by an affine function? The subspace $F = \operatorname{Vect}(1, x)$ is closed (finite-dimensional), and $p_F(f) = a + bx$ is characterized by orthogonality of the residual to $1$ and to $x$ :

\int_0^1(x^2 - a - bx)\,\dd x = 0, \qquad \int_0^1x\,(x^2 - a - bx)\,\dd x = 0,

i.e. $\frac13 = a + \frac b2$ and $\frac14 = \frac a2 + \frac b3$ : $a = -\frac16$ , $b = 1$ . So $p_F(x^2) = x - \frac16$ , and the error is

d(f, F)^2 = \int_0^1\Bigl(x^2 - x + \frac16\Bigr)^2\dd x = \frac1{180}, \qquad d(f, F) = \frac1{6\sqrt5} .

Two remarks worth internalizing. First, the computation is nothing but a $2\times2$ linear system — the normal equations; for the monomial basis their matrix $\bigl(\frac1{i+j+1}\bigr)$ is the notoriously ill-conditioned Hilbert matrix, and orthogonalizing first (Legendre polynomials, Problem 13.1) is the cure. Second, the best uniform approximation of $x^2$ by affine functions is different ( $x - \frac18$ , by equioscillation): each norm has its own geometry, and only the Hilbertian one answers with a linear system.

Read in context →

Examples

Examples

Related terms