What is orthonormal family?

Example 23.7 (Orthonormal coordinates, with a Parseval check)

Expand $x = (1, 2, 3)$ in the orthonormal basis of Exercise 23.3,

No system to solve — three inner products:

Certification by the norm formula of the proposition:

This sum-of-squared-coordinates check (a finite Parseval identity) costs seconds and catches sign and normalization errors with near certainty — make it a habit whenever an orthonormal expansion is computed; its infinite-dimensional version, for the Fourier coefficients of Example 23.14, is a theorem of the Year 3 volume.

Example 23.9 (Gram–Schmidt on polynomials, in full)

Orthonormalize $(1, X, X^2)$ in $\R_2[X]$ with $\langle P, Q\rangle = \int_0^1 PQ$. Step 1: $\norm{1}^2 = 1$, so $e_1 = 1$. Step 2: $w_2 = X - \langle X, 1\rangle\,1 = X - \frac12$, and $\norm{w_2}^2 = \int_0^1\bigl(x - \frac12\bigr)^2 \dd x = \frac1{12}$: $e_2 = \sqrt{12}\,\bigl(X - \frac12\bigr)$. Step 3: $\langle X^2, e_1\rangle = \frac13$ and

Its norm was computed in Exercise 23.9: $\norm{w_3}^2 = \frac1{180}$, whence $e_3 = \sqrt{180}\,\bigl(X^2 - X + \frac16\bigr)$. The polynomials $1$, $X - \frac12$, $X^2 - X + \frac16$ are, up to scale, the first Legendre polynomials of the interval $\intcc{0}{1}$; the construction continues one degree at a time, each new polynomial orthogonal to all its predecessors. Note how the algorithm recycles earlier work: the projection subtracted at step 3 is exactly the best affine approximation of $X^2$ found in Example 23.12 — Gram–Schmidt is iterated orthogonal projection.

Example 23.11 (Projections never lengthen)

Applying Pythagoras to the split $x = p_F(x) + (x - p_F(x))$:

with equality iff $x \in F$. In an orthonormal basis $(e_1, \dots, e_k)$ of $F$ this reads $\sum_{i \leq k}\langle x, e_i\rangle^2 \leq \norm x^2$ (a Bessel inequality): however many orthonormal directions one measures, the squared coordinates never exceed the squared length — compare the exact equality of Example 23.7 when the family is a full basis. This one-line inequality is what makes Fourier coefficients summable in the Year 3 volume; here it already explains why adding more basis functions to a least squares fit can only decrease the residual.