University Mathematics — Year 2 · Bachelor Year 2

12Quadratic Forms

A quadratic form is the algebraic shadow of a geometry: signature zero parts flatten, positive parts curve one way, negative the other. This chapter reduces every real quadratic form to a sum of $\pm$ squares (Gauss), proves that the counts of signs are intrinsic (Sylvester), and then crowns Euclidean geometry with the spectral theorem: symmetric endomorphisms diagonalize in orthonormal bases — the single most-used theorem of applied linear algebra.

12.1 Bilinear and quadratic forms

Definition 12.1

A symmetric bilinear form on a real vector space $E$ is a bilinear $\varphi \colon E \times E \to \R$ with $\varphi(x, y) = \varphi(y, x)$ ; the associated quadratic form is $q(x) = \varphi(x, x)$ . The form $\varphi$ is recovered from $q$ by polarization:

\varphi(x, y) = \tfrac12\bigl(q(x + y) - q(x) - q(y)\bigr).

In a basis $(e_i)$ , the matrix of $\varphi$ is the symmetric $B = (\varphi(e_i, e_j))$ , with $q(x) = X^{\mathsf T} B X$ ; a change of basis with matrix $P$ replaces $B$ by $P^{\mathsf T} B P$ (congruence — not similarity!). The rank of $q$ is $\operatorname{rk} B$ (invariant: congruence multiplies by invertible matrices).

Example 12.2

On $\R^2$ : $q(x, y) = x^2 + 4xy + y^2$ has matrix $\begin{pmatrix} 1 & 2\\ 2 & 1\end{pmatrix}$ . An inner product is exactly a symmetric bilinear form whose quadratic form is positive definite; this chapter studies the general, sign-indefinite case.

Example 12.3 (Congruence in action)

Take $q(x, y) = x^2 + 4xy + y^2$ (matrix $B = \begin{pmatrix} 1 & 2\\ 2 & 1\end{pmatrix}$ ) and the new basis $e_1' = (1, 1)$ , $e_2' = (1, -1)$ , i.e. $P = \begin{pmatrix} 1 & 1\\ 1 & -1\end{pmatrix}$ . Then

P^{\mathsf T}BP = \begin{pmatrix} 1 & 1\\ 1 & -1\end{pmatrix} \begin{pmatrix} 1 & 2\\ 2 & 1\end{pmatrix} \begin{pmatrix} 1 & 1\\ 1 & -1\end{pmatrix} = \begin{pmatrix} 6 & 0\\ 0 & -2\end{pmatrix} :

in the coordinates $(u, v)$ along the new basis, $q = 6u^2 - 2v^2$ — check: $x = u + v$ , $y = u - v$ gives $x^2 + 4xy + y^2 = 6u^2 - 2v^2$ directly. Note the new diagonal entries $6, -2$ are not the eigenvalues $3, -1$ of $B$ : congruence rescales, only similarity preserves spectra — but the signs agree, as Sylvester’s theorem demands. (The basis here is orthogonal but not orthonormal; normalizing it by $\frac{1}{\sqrt2}$ would divide the diagonal by $2$ and recover the eigenvalues.)

Example 12.4 (Gram determinants measure area)

For $v_1, v_2$ in a Euclidean space, the Gram matrix $G = \bigl(\langle v_i, v_j\rangle\bigr)$ packages lengths and angle; its determinant packages area:

\det G = \norm{v_1}^2\norm{v_2}^2 - \langle v_1, v_2\rangle^2 = \norm{v_1}^2\norm{v_2}^2\bigl(1 - \cos^2\theta\bigr) = \bigl(\norm{v_1}\,\norm{v_2}\sin\theta\bigr)^2 ,

the squared area of the parallelogram on $v_1, v_2$ — and Cauchy–Schwarz is exactly the statement $\det G \geq 0$ . Worked instance: $v_1 = (1, 2, 2)$ , $v_2 = (2, 1, -2)$ in $\R^3$ :

G = \begin{pmatrix} 9 & 0\\ 0 & 9 \end{pmatrix}, \qquad \det G = 81 :

the vectors are orthogonal of length $3$ , spanning a parallelogram (here, a square) of area $\sqrt{81} = 9$ . Closing insight: no cross product and no dimension- $3$ magic was used — $\sqrt{\det G}$ measures $k$ -dimensional volume in any dimension, the starting point of the weekend problem’s Part I and of the surface-area integrals later in this volume.

12.2 Gauss reduction and Sylvester’s inertia

Theorem 12.5 (Gauss reduction)

Every quadratic form $q$ on a finite-dimensional real space can be written

q = \sum_{i=1}^{s} \ell_i^2 - \sum_{j=1}^{t} m_j^2 ,

where $\ell_1, \dots, \ell_s, m_1, \dots, m_t$ are linearly independent linear forms; equivalently, some basis makes the matrix of $q$ diagonal with entries $+1$ ( $s$ times), $-1$ ( $t$ times), $0$ .

Proof. Induction on the number of variables, in coordinates: $q(x_1, \dots, x_n)$ .

Case 1: some square appears, say the coefficient $a$ of $x_1^2$ is nonzero. Group all the $x_1$ -terms and complete the square:

q = a\Bigl(x_1 + \frac{1}{a}\,\lambda(x_2, \dots, x_n)\Bigr)^{\!2} + q_1(x_2, \dots, x_n),

where $\lambda$ is linear and $q_1$ quadratic in the remaining variables: one independent form split off (it involves $x_1$ , the others do not), induction applies to $q_1$ , and $\pm$ signs come from the sign of $a$ after rescaling by $\sqrt{\abs a}$ .

Case 2: no square, but some cross term, say $b\,x_1x_2$ with $b \neq 0$ . Use the identity

x_1x_2 = \tfrac14\bigl((x_1 + x_2)^2 - (x_1 - x_2)^2\bigr)

after grouping: writing $q = b\,x_1x_2 + x_1\alpha + x_2\beta + q_2$ (with $\alpha, \beta, q_2$ in the other variables), one checks

q = \frac{b}{4}\Bigl[\Bigl(x_1 + x_2 + \frac{\alpha + \beta}{b}\Bigr)^{2} - \Bigl(x_1 - x_2 + \frac{\beta - \alpha}{b}\Bigr)^{2}\Bigr] + \widetilde q ,

with $\widetilde q$ free of $x_1, x_2$ : two independent forms split off, and induction finishes.

Independence of the collected forms: order the batches as produced. The forms of the first batch contain $x_1$ (Case 1) or $x_1, x_2$ (Case 2); all later forms are free of those variables. Suppose a linear combination of all collected forms vanishes. Reading the coefficient of $x_1$ (and $x_2$ ): only the first batch contributes, and within that batch the one or two forms are visibly independent ( $\ell$ alone; or $\ell \pm m$ with independent $\ell, m$ ): the first-batch coefficients vanish. Strip the batch and repeat: by induction along the batches, all coefficients vanish — the whole family is free, triangularity made explicit. ∎

Theorem 12.6 (Sylvester’s law of inertia)

The pair $(s, t)$ in Theorem 12.5 depends only on $q$ , not on the reduction: it is the signature of $q$ . Moreover

s = \max\{\dim F : q|_F \text{ positive definite}\},

and symmetrically for $t$ .

Proof. Let $q = \sum_{i \leq s}\ell_i^2 - \sum_{j\leq t} m_j^2$ and let $F_+$ be the span of the (pre-)dual vectors on which $(\ell_i)$ restrict to coordinates — concretely: complete the independent family $(\ell_1, \dots, \ell_s, m_1, \dots, m_t)$ into a basis of the dual $E^*$ , and let $(u_1, \dots, u_n)$ be the basis of $E$ whose coordinate forms these are (the pre-dual basis: $\ell_i(u_k) = \delta_{ik}$ for $k \leq s$ , and the later forms vanish on the earlier vectors). Set $F_+ = \operatorname{Vect}(u_1, \dots, u_s)$ : for $x = \sum_{i\leq s}x_iu_i \in F_+$ ,

\ell_i(x) = x_i, \qquad m_j(x) = 0, \qquad\text{so}\qquad q(x) = \sum_{i\leq s}x_i^2 > 0 \quad (x \neq 0) :

$q|_{F_+}$ is positive definite and the max in the display is $\geq s$ . Conversely, let $F$ be any subspace with $q|_F$ positive definite, and $G = \{x : \ell_1(x) = \dots = \ell_s(x) = 0\}$ , of codimension $\leq s$ ; on $G$ , $q(x) = -\sum m_j^2 \leq 0$ . Then $F \cap G = \{0\}$ (a nonzero vector there would have $q > 0$ and $q \leq 0$ ), so $\dim F \leq \dim E - \dim G \leq s$ . Hence the max equals $s$ for every reduction: $s$ is intrinsic, and $t = \operatorname{rk} q - s$ likewise. ∎

Example 12.7

$q(x, y, z) = xy + yz + zx$ (no squares). With $x_1 = x$ , $x_2 = y$ : $q = xy + z(x + y)$ , and the two-square identity gives

q = \tfrac14(x + y + 2z)^2 - \tfrac14(x - y)^2 - z^2 ,

(expand to check). Three independent forms: signature $(1, 2)$ , rank $3$ . One positive direction, two negative: the “light cone” geometry of this form.

Example 12.8 (A degenerate form, reduced in full)

$q(x, y, z) = xy + yz$ on $\R^3$ : no squares, so Case 2 with the grouping $q = y(x + z)$ . The two-square identity on the product of the independent forms $y$ and $x + z$ :

q = \frac14\bigl(y + x + z\bigr)^2 - \frac14\bigl(y - x - z\bigr)^2 .

The two linear forms $y + x + z$ and $y - x - z$ are independent (their difference is $2(x+z)$ , their sum $2y$ ), so Sylvester reads off: signature $(1, 1)$ , rank $2$ — degenerate. The kernel of the polar form is found by solving $\varphi(v, \cdot) = 0$ : with matrix $\frac12\begin{pmatrix} 0&1&0\\ 1&0&1\\ 0&1&0\end{pmatrix}$ , the kernel is $\{y = 0,\ x + z = 0\} = \R\,(1, 0, -1)$ , the direction along which $q$ sees nothing. Closing insight: rank deficiency shows up in Gauss as “running out of variables” — the reduction produced only two squares out of three dimensions, and the missing dimension is exactly the kernel.

Example 12.9 (One form, two roads to the signature)

$q(x, y, z) = 2x^2 + 2y^2 + 2z^2 + 2xy + 2yz$ , matrix $\begin{pmatrix} 2 & 1 & 0\\ 1 & 2 & 1\\ 0 & 1 & 2 \end{pmatrix}$ . Road 1, Gauss: complete squares in order,

q = 2\Bigl(x + \frac y2\Bigr)^{\!2} + \frac32 y^2 + 2yz + 2z^2 = 2\Bigl(x + \frac y2\Bigr)^{\!2} + \frac32\Bigl(y + \frac{2z}{3}\Bigr)^{\!2} + \frac43 z^2 :

three positive squares on independent forms, signature $(3, 0)$ : positive definite. Road 2, eigenvalues: the matrix is the tridiagonal $2I + N$ with $N$ the neighbor matrix; its eigenvalues are $2 + \sqrt2$ , $2$ , $2 - \sqrt2$ (check the eigenvectors $(1, \pm\sqrt2, 1)$ and $(1, 0, -1)$ ), all positive: same verdict, by Corollary 12.15. Gauss is faster; eigenvalues say more (they give the principal axes and the extreme values of $q$ on the sphere). Closing insight: the positive pivots $2, \frac32, \frac43$ of Gauss are exactly the ratios $\frac{\Delta_k}{\Delta_{k-1}}$ of leading principal minors ( $\Delta_1 = 2$ , $\Delta_2 = 3$ , $\Delta_3 = 4$ ) — the weekend problem proves this in general.

Method 12.10 (Computing a signature: three routes)

Gauss (always works, fastest by hand): complete squares in order, Case 2 when no square is available; count the signs. Check that the collected linear forms are independent — fewer forms than variables means a kernel (Example 12.8).
Leading minors (for definiteness tests): all $\Delta_k > 0$ iff positive definite (Exercise 12.8); the pivots $\Delta_k/\Delta_{k-1}$ even give the Gauss coefficients (weekend problem). Fails silently if some $\Delta_k = 0$ : fall back on route 1.
Eigenvalues (most informative, most expensive): signs of the spectrum (Corollary 12.15); also yields the principal axes and the extreme values of $q$ on the unit sphere. Prefer it when the eigenstructure is needed anyway.

12.3 The spectral theorem

Now let $E$ be Euclidean (inner product $\langle\cdot,\cdot\rangle$ , Year 1 volume).

Definition 12.11 (Adjoint; symmetric endomorphisms)

For $u \in \mathcal{L}(E)$ , the adjoint $u^*$ is the unique endomorphism with

\langle u(x), y\rangle = \langle x, u^*(y)\rangle \qquad (x, y \in E);

in an orthonormal basis, $\operatorname{Mat}(u^*) = \operatorname{Mat}(u)^{\mathsf T}$ . $u$ is symmetric (self-adjoint) when $u^* = u$ — equivalently its matrix in an orthonormal basis is symmetric.

Existence and uniqueness of the adjoint. For fixed $y$ , the form $x \mapsto \langle u(x), y\rangle$ is linear, hence (finite dimension) of the shape $\langle x, z_y\rangle$ for a unique $z_y$ — the map $y \mapsto z_y =: u^*(y)$ being linear by uniqueness. The matrix identification: $\langle u(e_i), e_j\rangle$ read both ways. ∎

Example 12.12 (The adjoint depends on the inner product)

On $\R^2$ take the weighted inner product $\langle x, y\rangle_D = x_1y_1 + 2x_2y_2$ (matrix $D = \operatorname{diag}(1,2)$ ) and $u$ of matrix $A = \begin{pmatrix} 0 & 1\\ 0 & 0\end{pmatrix}$ in the canonical basis. From $\langle u(x), y\rangle_D = (Ax)^{\mathsf T}Dy = x^{\mathsf T}(A^{\mathsf T}D)y$ and $\langle x, u^*(y)\rangle_D = x^{\mathsf T}(DA^*)y$ , the adjoint’s matrix is

A^* = D^{-1}A^{\mathsf T}D = \begin{pmatrix} 1 & 0\\ 0 & \tfrac12\end{pmatrix} \begin{pmatrix} 0 & 0\\ 1 & 0\end{pmatrix} \begin{pmatrix} 1 & 0\\ 0 & 2\end{pmatrix} = \begin{pmatrix} 0 & 0\\ \tfrac12 & 0\end{pmatrix} \neq A^{\mathsf T} .

Sanity check on $x = (1,0)$ , $y = (0,1)$ :

\langle u(x), y\rangle_D = \langle (0,0), y\rangle_D = 0, \quad \langle x, u^*(y)\rangle_D = \langle(1,0), (0,\tfrac12)\rangle_D = 0 ;

on $x = (0,1)$ , $y = (1,0)$ :

\langle u(x), y\rangle_D = \langle(1,0),(1,0)\rangle_D = 1, \quad \langle x, u^*(y)\rangle_D = \langle(0,1),(0,\tfrac12)\rangle_D = 1 .

Closing insight: “ $\operatorname{Mat}(u^*) = \operatorname{Mat}(u)^{\mathsf T}$ ” is a statement about orthonormal bases only; in general the metric $D$ intervenes, exactly as in the simultaneous reduction of the weekend problem.

Theorem 12.13 (Spectral theorem)

Let $u$ be a symmetric endomorphism of a Euclidean space $E$ . Then $E$ has an orthonormal basis of eigenvectors of $u$ ; all eigenvalues are real, and eigenspaces for distinct eigenvalues are orthogonal. Matrix form: every real symmetric matrix $A$ writes

A = P\,D\,P^{\mathsf T}, \qquad P \text{ orthogonal } (P^{\mathsf T} P = I),\ D \text{ diagonal}.

Proof. An eigenvector exists. The function $x \mapsto \langle u(x), x\rangle$ is continuous on the unit sphere $S$ of $E$ , which is compact (finite dimension, Theorem 5.13): it attains its maximum $\lambda$ at some $a \in S$ . We claim $u(a) = \lambda a$ . For any $y \perp a$ with $\norm y = 1$ and $t \in \R$ , the vector $x_t = \frac{a + ty}{\sqrt{1 + t^2}}$ lies on $S$ ( $\norm{a + ty}^2 = 1 + t^2$ by Pythagoras); expanding the maximized function,

g(t) = \langle u(x_t), x_t\rangle = \frac{\langle u(a), a\rangle + 2t\langle u(a), y\rangle + t^2\langle u(y), y\rangle}{1 + t^2}

(symmetry of $u$ merged the two cross terms: $\langle u(a), y\rangle = \langle a, u(y)\rangle = \langle u(y), a\rangle$ ). $g$ is a differentiable function of $t$ with a maximum at $t = 0$ ; the quotient rule at $0$ gives

g'(0) = \frac{2\langle u(a), y\rangle\cdot 1 - \langle u(a), a\rangle\cdot 0}{1} = 2\langle u(a), y\rangle = 0 .

So $u(a)$ is orthogonal to the whole hyperplane $a^\perp$ : $u(a) \in (a^{\perp})^{\perp} = \R a$ , i.e. $u(a) = \mu a$ ; and $\mu = \langle u(a), a\rangle = \lambda$ .

Induction. The orthogonal complement $F = a^\perp$ is $u$ -stable: for $x \perp a$ , $\langle u(x), a\rangle = \langle x, u(a)\rangle = \lambda\langle x, a\rangle = 0$ . The restriction $u|_F$ is symmetric for the induced inner product; by induction on the dimension, $F$ has an orthonormal eigenbasis; prepend $a$ .

Complements. Eigenvalues are the real numbers $\langle u(e), e\rangle$ on the eigenbasis. Orthogonality of eigenspaces: $u(x) = \lambda x$ , $u(y) = \mu y$ give $\lambda\langle x, y\rangle = \langle u(x), y\rangle = \langle x, u(y)\rangle = \mu\langle x, y\rangle$ , so $\langle x, y\rangle = 0$ when $\lambda \neq \mu$ . Matrix form: columns of $P$ = the orthonormal eigenbasis. ∎

Example 12.14 (A full spectral run)

Diagonalize orthogonally $A = \begin{pmatrix} 1 & 2\\ 2 & 1\end{pmatrix}$ . Characteristic polynomial $(1 - \lambda)^2 - 4$ : eigenvalues $3$ and $-1$ . Eigenvectors: $(A - 3I)v = 0$ gives $v_1 = \frac{1}{\sqrt2}(1,1)$ ; $(A + I)v = 0$ gives $v_2 = \frac{1}{\sqrt2}(1,-1)$ — orthogonal, as Theorem 12.13 guarantees without computation. With $P = (v_1\ v_2)$ (a rotation by $\frac\pi4$ ):

P^{\mathsf T}AP = \begin{pmatrix} 3 & 0\\ 0 & -1 \end{pmatrix}, \qquad x^2 + 4xy + y^2 = 3u^2 - v^2 \quad\text{in the rotated frame} .

So the form of Exercise 12.1 is a hyperbola-type form: signature $(1,1)$ , consistent with its Gauss reduction $(x + 2y)^2 - 3y^2$ — different squares, same signature, as Sylvester demands. Closing insight: Gauss gave the answer faster, but the spectral route also reports that on the unit circle $q$ ranges exactly over $\intcc{-1}{3}$ , attained along $v_2$ and $v_1$ : the extra work buys geometry.

Corollary 12.15 (Principal axes; positivity tests)

Every quadratic form $q$ on a Euclidean space diagonalizes in some orthonormal basis: $q(x) = \sum_i \lambda_i x_i^2$ with $\lambda_i$ the eigenvalues of the symmetric matrix of $q$ ; the signature counts positive and negative eigenvalues.
A symmetric matrix is positive semidefinite (resp. definite) iff all its eigenvalues are $\geq 0$ (resp. $> 0$ ); and then the extreme values of the Rayleigh quotient are

$\min_{\norm x = 1} \langle Ax, x\rangle = \lambda_{\min}, \qquad \max_{\norm x = 1} \langle Ax, x\rangle = \lambda_{\max} .$

Proof. (1) Write $q(x) = \langle A x, x\rangle$ with $A$ symmetric (the matrix of $q$ in an orthonormal basis); diagonalize $A$ by the spectral theorem: for $x = \sum x_ie_i$ in the orthonormal eigenbasis,

q(x) = \Bigl\langle \sum_i \lambda_ix_ie_i,\ \sum_j x_je_j\Bigr\rangle = \sum_i \lambda_i x_i^2

(orthonormality kills the cross terms). The $\pm$ signs of the $\lambda_i$ count the signature by Sylvester: rescaling each coordinate by $\sqrt{\abs{\lambda_i}}$ exhibits a Gauss reduction with independent forms.

(2) In the eigenbasis, $\langle Ax, x\rangle = \sum \lambda_i x_i^2$ , trapped between $\lambda_{\min}\norm x^2$ and $\lambda_{\max}\norm x^2$ , with equality at the corresponding eigenvectors; positivity of all eigenvalues is thus equivalent to positivity of the form. ∎

Example 12.16

$A = \begin{pmatrix} 2 & 1\\ 1 & 2 \end{pmatrix}$ : eigenvalues $3$ (eigenvector $\frac{1}{\sqrt2}(1,1)$ ) and $1$ ( $\frac{1}{\sqrt2}(1,-1)$ ). The quadratic form $2x^2 + 2xy + 2y^2$ becomes $3X^2 + Y^2$ in the rotated orthonormal frame: an ellipse’s principal axes, computed. The Gauss reduction reaches a diagonal form too, but only the spectral theorem reaches it without distorting lengths.

Example 12.17 (An ellipse fully identified)

Which curve is $5x^2 + 4xy + 2y^2 = 6$ ? The matrix $\begin{pmatrix} 5 & 2\\ 2 & 2\end{pmatrix}$ has characteristic polynomial $\lambda^2 - 7\lambda + 6 = (\lambda - 1)(\lambda - 6)$ : eigenvalues $1$ and $6$ , both positive — an ellipse. Orthonormal eigenvectors: for $\lambda = 1$ , solve $\begin{pmatrix} 4 & 2\\ 2 & 1\end{pmatrix}v = 0$ : $v_1 = \frac{1}{\sqrt5}(1, -2)$ ; for $\lambda = 6$ : $v_2 = \frac{1}{\sqrt5}(2, 1)$ . In the rotated coordinates $(X, Y)$ along $(v_2, v_1)$ the equation becomes

6X^2 + Y^2 = 6, \qquad\text{i.e.}\qquad X^2 + \frac{Y^2}{6} = 1 :

semi-axes $1$ (along $v_2$ ) and $\sqrt6$ (along $v_1$ ). Closing insight: the rough shape was free of charge — $\det = 6 > 0$ and positive trace announce an ellipse before any eigenvector is computed — but only the spectral theorem delivers the axes’ directions and lengths, i.e. the actual geometry.

Example 12.18 (Extremes on the sphere, read off the spectrum)

What are the extreme values of $q(x,y,z) = 2xy + 2yz + 2zx$ on the unit sphere? Its matrix (the all-ones off-diagonal of Exercise 12.2) has eigenvalues $2$ and $-1$ (double), so by Corollary 12.15 (2):

\max_{\norm v = 1} q(v) = 2 \ \text{ at } v = \tfrac{1}{\sqrt3}(1,1,1), \qquad \min_{\norm v = 1} q(v) = -1 \ \text{ on the circle } x + y + z = 0 .

No calculus, no Lagrange multipliers: the spectral theorem solves this constrained optimization outright — and exhibits the maximizer. Closing insight: compare with the multiplier method of the differential-calculus chapter, which finds the same critical points with more work; for quadratic objectives on spheres, spectra are the royal road (the weekend problem of the Hermitian chapter builds the whole Courant–Fischer theory on this observation).

Remark 12.19 (Common pitfalls)

(i) Congruence is not similarity: a change of basis for a form acts by $P^{\mathsf T}BP$ , not $P^{-1}BP$ ; eigenvalues are not invariants of a quadratic form ( $I$ and $4I$ are congruent via $P = 2I$ ) — only their signs are (Sylvester). Speak of the eigenvalues of a form only once an inner product is fixed. (ii) Positive entries prove nothing: $\begin{pmatrix} 1 & 2\\ 2 & 1\end{pmatrix}$ has all entries positive yet signature $(1,1)$ ( $\det = -3$ ); conversely a positive definite matrix may have negative off-diagonal entries (Example 12.9 shifted: $2I - N$ works equally). Use Method 12.10. (iii) Dependent squares: writing $q = \ell_1^2 - \ell_2^2$ says nothing if $\ell_1, \ell_2$ are proportional — $x^2 + 2xy + y^2 = (x+y)^2$ has rank $1$ , not $2$ ; always check independence before reading off the signature. (iv) Sphere extremes without compactness: the Rayleigh bounds of Corollary 12.15 are attained because the sphere is compact; on the open ball or the whole space, an indefinite form has neither maximum nor minimum.

Remark 12.20 (Where this is used)

The spectral theorem is the single most exported result of this book: statistics diagonalizes covariance matrices with it (principal component analysis), mechanics extracts normal modes of oscillation from it (the simultaneous reduction of the weekend problem), numerical analysis builds Cholesky and singular value decompositions on it (same problem), and the next chapter transports it to complex Hermitian spaces. The Year 3 volume proves its infinite-dimensional avatar for compact self-adjoint operators, where the finite-dimensional proof’s compactness argument becomes the whole story.

Remark 12.21 (Perspectives within this volume)

Quadratic forms thread through the rest of Book 4 under three disguises. As Hessians: the differential-calculus chapter classifies critical points by the signature of the second-order form, so Sylvester’s invariance is what makes “saddle” a well-defined word. As energies: the differential-equations chapter’s oscillators carry the quadratic energy $\frac12x'^2 + \frac12\omega^2x^2$ , and the simultaneous reduction of this chapter’s weekend problem is exactly the extraction of normal modes. As geometry: the conics of this chapter grow into the quadric surfaces of the geometry chapters, where the second fundamental form of a surface — a quadratic form on each tangent plane — has signature deciding whether the surface curves like a bowl or a saddle. The Hermitian chapter, next, replays the whole score over $\C$ .

12.4 Exercises

Exercise 12.1 ★

Gauss-reduce and give rank and signature:

q_1(x,y) = x^2 + 4xy + y^2, \qquad q_2(x,y,z) = x^2 + 2y^2 + 3z^2 + 2xy + 2yz .

Solution

Solution of Exercise 12.1.

$q_1 = (x + 2y)^2 - 3y^2$ : rank $2$ , signature $(1, 1)$ (a hyperbola-type form).

$q_2$ : complete the square in $x$ : $q_2 = (x + y)^2 + y^2 + 2yz + 3z^2 = (x+y)^2 + (y + z)^2 + 2z^2$ : rank $3$ , signature $(3, 0)$ — positive definite.

Exercise 12.2 ★

Diagonalize orthogonally $A = \begin{pmatrix} 0 & 1 & 1\\ 1 & 0 & 1\\ 1 & 1 & 0\end{pmatrix}$ (eigenvalues from Chapter 3’s computation; now make the basis orthonormal) and reduce the form $q(x,y,z) = 2xy + 2yz + 2zx$ to principal axes.

Solution

Solution of Exercise 12.2.

Eigenvalues $2$ (on $\operatorname{Vect}(1,1,1)$ ) and $-1$ (on the plane $x + y + z = 0$ ). Orthonormalize: $e_1 = \frac{1}{\sqrt3}(1,1,1)$ ; in the plane, Gram–Schmidt on $(1,-1,0), (1,0,-1)$ gives $e_2 = \frac{1}{\sqrt2}(1,-1,0)$ , $e_3 = \frac{1}{\sqrt6}(1,1,-2)$ . Then $P = (e_1\ e_2\ e_3)$ is orthogonal with $P^{\mathsf T}AP = \operatorname{diag}(2,-1,-1)$ .

The form $q = 2xy + 2yz + 2zx$ has matrix $A$ : in the rotated coordinates $q = 2X^2 - Y^2 - Z^2$ — principal axes; signature $(1,2)$ , agreeing with Example 12.7 (the same form!).

Exercise 12.3 ★

Prove that $u^{**} = u$ , $(u \circ v)^* = v^* \circ u^*$ , and that $\ker u^* = (\operatorname{im} u)^{\perp}$ . Deduce $\operatorname{rk} u^* = \operatorname{rk} u$ .

Solution

Solution of Exercise 12.3.

$u^{**} = u$ : $\langle u^{**}x, y\rangle = \langle x, u^*y\rangle = \langle ux, y\rangle$ for all $y$ . $(uv)^* = v^*u^*$ : $\langle uvx, y\rangle = \langle vx, u^*y\rangle = \langle x, v^*u^*y\rangle$ . Kernel: $y \in \ker u^* \iff \langle x, u^*y \rangle = 0\ \forall x \iff \langle u(x), y\rangle = 0\ \forall x \iff y \perp \operatorname{im} u$ . Ranks: $\dim\ker u^* = n - \operatorname{rk} u$ (orthogonal complement), so $\operatorname{rk} u^* = \operatorname{rk} u$ by rank–nullity — the Euclidean avatar of the transpose-rank theorem.

Exercise 12.4 ★★

Let $A$ be real symmetric with $A^3 = A$ . Prove that $A^2$ is the matrix of an orthogonal projection. More generally, relate the spectral decompositions of $A$ and $P(A)$ for a polynomial $P$ .

Solution

Solution of Exercise 12.4.

Spectral: $A = PDP^{\mathsf T}$ , $D$ diagonal with entries $\lambda_i$ satisfying $\lambda_i^3 = \lambda_i$ : $\lambda_i \in \{-1, 0, 1\}$ . Then $A^2 = PD^2P^{\mathsf T}$ with $D^2$ diagonal of entries $0/1$ : $A^2$ is symmetric and idempotent ( $(A^2)^2 = A^4 = A\cdot A^3 = A^2$ ) — symmetric idempotent $=$ orthogonal projection (it is the projection onto $\ker(A^2 - I) = \ker(A-I)\oplus\ker(A+I)$ along $\ker A$ , and these are orthogonal by the spectral theorem).

Generally, $P(A) = P\!\left(\text{diag}\right)$ : $P(A)$ has the same eigenvectors, eigenvalues $P(\lambda_i)$ — the “spectral mapping” at the diagonalizable level.

Exercise 12.5 ★★

Prove that $O(n) = \{P : P^{\mathsf T}P = I\}$ is a compact subset of $\mathcal{M}_n(\R)$ (closed: preimage of $\{I\}$ under a continuous map; bounded: columns are unit vectors). Is it connected?

Solution

Solution of Exercise 12.5.

Closed: $O(n) = g^{-1}(\{I\})$ for the continuous $g(P) = P^{\mathsf T}P$ (polynomial entries). Bounded: each column of $P \in O(n)$ is a unit vector, so all entries lie in $\intcc{-1}{1}$ . Closed and bounded in $\mathcal{M}_n(\R) \simeq \R^{n^2}$ : compact (Theorem 4.16 (2)).

Not connected: $\det$ takes the two values $\pm1$ on $O(n)$ , and a continuous surjection onto $\{-1, 1\}$ splits the space (Example 4.28’s argument).

Exercise 12.6 ★★

(Square root) Let $A$ be symmetric positive semidefinite. Construct a symmetric positive semidefinite $B$ with $B^2 = A$ , and prove that it is unique (existence: take square roots of the eigenvalues in a spectral basis; uniqueness: a candidate $B$ commutes with $A = B^2$ , so preserves its eigenspaces — reduce to the scalar case on each).

Solution

Solution of Exercise 12.6.

Existence: $A = PDP^{\mathsf T}$ with $D = \operatorname{diag}(\lambda_i)$ , $\lambda_i \geq 0$ ; set $B = P\sqrt D P^{\mathsf T}$ with $\sqrt D = \operatorname{diag}(\sqrt{\lambda_i})$ : symmetric, positive semidefinite, $B^2 = A$ .

Uniqueness: let $B$ be symmetric psd with $B^2 = A$ . $B$ commutes with $A$ ; hence $B$ preserves each eigenspace $E_\lambda(A)$ (for $Ax = \lambda x$ : $A(Bx) = BAx = \lambda Bx$ ). On $E_\lambda(A)$ , the restriction of $B$ is symmetric psd with square $\lambda\,\mathrm{id}$ ; its eigenvalues $\mu$ satisfy $\mu^2 = \lambda$ , $\mu \geq 0$ : $\mu = \sqrt\lambda$ — so the restriction, being diagonalizable with the single eigenvalue $\sqrt\lambda$ , is $\sqrt\lambda\,\mathrm{id}$ . Since $E = \bigoplus E_\lambda(A)$ , $B$ is determined: $B = \sqrt A$ .

Exercise 12.7 ★★

For real symmetric $A$ , prove that $\vertiii{A}_2 := \sup_{\norm x_2 = 1}\norm{Ax}_2 = \max_i \abs{\lambda_i}$ (spectral radius), and compute $\vertiii{A}_2$ for $A = \begin{pmatrix} 1 & 2\\ 2 & 1\end{pmatrix}$ .

Solution

Solution of Exercise 12.7.

In an orthonormal eigenbasis, $\norm{Ax}_2^2 = \sum \lambda_i^2 x_i^2 \leq (\max_i \lambda_i^2)\norm x_2^2$ , with equality at the corresponding eigenvector: $\vertiii A_2 = \max\abs{\lambda_i}$ . For the given matrix: eigenvalues $3, -1$ (Example 12.16’s twin): $\vertiii A_2 = 3$ .

Exercise 12.8 ★★★

(Sylvester’s criterion) Let $A$ be real symmetric with leading principal minors $\Delta_1, \dots, \Delta_n$ (determinants of the top-left blocks). Prove that $A$ is positive definite if and only if all $\Delta_k > 0$ . (For $\Rightarrow$ : restrictions of a definite form are definite, and the determinant of a positive definite matrix — the product of its eigenvalues — is positive. For $\Leftarrow$ : induct on $n$ ; the top-left $(n-1)$ -block is positive definite, diagonalize the form on that subspace and complete the square in the last variable; the sign of the last diagonal entry is governed by $\det A = \Delta_n > 0$ .)

Solution

Solution of Exercise 12.8.

( $\Rightarrow$ ) The top-left $k \times k$ block $A_k$ is the matrix of the restriction of the (definite) form to the span of the first $k$ basis vectors: positive definite, so its eigenvalues are positive and $\Delta_k = \det A_k > 0$ .

( $\Leftarrow$ ) Induction on $n$ ; $n = 1$ clear. Assume all $\Delta_k > 0$ . By induction, $A_{n-1}$ is positive definite: the form $q$ restricted to $F = \operatorname{Vect}(e_1, \dots, e_{n-1})$ is definite. Diagonalize $q|_F$ (Gauss): coordinates $y_1, \dots, y_{n-1}$ with $q|_F = \sum y_i^2$ . In the full space, completing the square in the last variable,

q = \sum_{i=1}^{n-1} \bigl(y_i + c_i x_n\bigr)^2 + c\,x_n^2

for suitable constants (collect the cross terms into the squares). The reduction exhibits signature $(n-1 + \epsilon, \cdot)$ with $\epsilon$ the sign contribution of $c$ ; and the determinant keeps the sign of the product of the diagonal coefficients under congruence ( $\det(P^{\mathsf T}AP) = (\det P)^2\det A$ ): $\Delta_n > 0$ forces $c > 0$ . Hence $q$ is a sum of $n$ squares of independent forms: positive definite.

Exercise 12.9 ★★★

(Courant–Fischer, second eigenvalue) Let $u$ be symmetric with eigenvalues $\lambda_1 \geq \lambda_2 \geq \dots \geq \lambda_n$ . Prove

\lambda_2 = \min_{\substack{H \text{ hyperplane}}}\; \max_{\substack{x \in H,\ \norm x = 1}} \langle u(x), x\rangle .

(For $\leq$ : any hyperplane meets the $2$ -plane spanned by the top two eigenvectors. For $\geq$ : choose $H = (e_1)^{\perp}$ .)

Solution

Solution of Exercise 12.9.

Let $(e_1, \dots, e_n)$ be an orthonormal eigenbasis for $\lambda_1 \geq \dots \geq \lambda_n$ .

$\lambda_2 \leq$ the min-max: for any hyperplane $H$ , the $2$ -dimensional $V = \operatorname{Vect}(e_1, e_2)$ satisfies $\dim(H \cap V) \geq 1$ (Grassmann): pick a unit $x \in H \cap V$ , $x = ae_1 + be_2$ , $a^2 + b^2 = 1$ :

\langle u(x), x\rangle = \lambda_1 a^2 + \lambda_2 b^2 \geq \lambda_2 :

every hyperplane’s max is $\geq \lambda_2$ .

$\geq$ : for $H = e_1^{\perp}$ , every unit $x = \sum_{i\geq2} x_ie_i \in H$ has $\langle u(x), x\rangle = \sum_{i \geq 2} \lambda_i x_i^2 \leq \lambda_2$ , attained at $e_2$ : this hyperplane’s max is exactly $\lambda_2$ . The min over $H$ is therefore $\lambda_2$ .

Exercise 12.10 ★★

Determine rank and signature of $q(x_1, \dots, x_n) = \sum_{i < j} x_ix_j$ on $\R^n$ ( $n \geq 2$ ), in two ways: by the algebraic identity $2q = \bigl(\sum x_i\bigr)^2 - \sum x_i^2$ together with the restriction of $q$ to the hyperplane $\sum x_i = 0$ ; and by computing the eigenvalues of its matrix $\frac12(J - I)$ , where $J$ is the all-ones matrix.

Solution

Solution of Exercise 12.10.

Algebraic road: $2q = \bigl(\sum x_i\bigr)^2 - \sum x_i^2$ . On the hyperplane $H : \sum x_i = 0$ (dimension $n - 1$ ), $q = -\frac12\sum x_i^2$ is negative definite; on the line $\R(1, \dots, 1)$ , $q(t, \dots, t) = \binom n2 t^2 > 0$ . A subspace where $q$ is positive definite meets $H$ trivially, so has dimension $\leq 1$ : by Sylvester (Theorem 12.6), $s = 1$ , and $t \geq n-1$ from $H$ ; rank $\leq n$ forces signature $(1, n-1)$ , rank $n$ .

Spectral road: the matrix is $\frac12(J - I)$ ; $J$ has eigenvalues $n$ (on $(1,\dots,1)$ ) and $0$ (on $H$ ), so $\frac12(J-I)$ has eigenvalues $\frac{n-1}{2}$ (once) and $-\frac12$ ( $n-1$ times): one positive, $n-1$ negative — same signature, by Corollary 12.15.

Exercise 12.11 ★★

Let $A$ , $B$ be real symmetric with $B$ positive semidefinite. Prove

\lambda_{\min}(A)\operatorname{tr} B \;\leq\; \operatorname{tr}(AB) \;\leq\; \lambda_{\max}(A)\operatorname{tr} B .

(Write $B = C^{\mathsf T}C$ and $\operatorname{tr}(AB) = \sum_i \langle A c_i, c_i\rangle$ over the columns $c_i$ of $C^{\mathsf T}$ .) In particular $\operatorname{tr}(AB) \geq 0$ when both are positive semidefinite.

Solution

Solution of Exercise 12.11.

Write $B = C^{\mathsf T}C$ (Exercise 12.6 via $C = \sqrt B$ ). Then, with $c_1, \dots, c_n$ the columns of $C^{\mathsf T}$ :

\operatorname{tr}(AB) = \operatorname{tr}(AC^{\mathsf T}C) = \operatorname{tr}(CAC^{\mathsf T}) = \sum_{i=1}^n \langle A c_i, c_i\rangle .

By Corollary 12.15 (2), each term lies between $\lambda_{\min}(A)\norm{c_i}^2$ and $\lambda_{\max}(A)\norm{c_i}^2$ , and $\sum\norm{c_i}^2 = \operatorname{tr}(C^{\mathsf T}C)^{\vphantom1} = \operatorname{tr} B$ : the double inequality follows. If $A$ is also positive semidefinite, $\lambda_{\min}(A) \geq 0$ : $\operatorname{tr}(AB) \geq 0$ .

Exercise 12.12 ★★★

On $E = \mathcal{M}_n(\R)$ , consider $q(M) = \operatorname{tr}(M^2)$ .

Show that $q$ is a quadratic form with polar form $\varphi(M, N) = \operatorname{tr}(MN)$ .
Show that the symmetric and antisymmetric matrices form $\varphi$ -orthogonal subspaces on which $q$ is respectively positive definite and negative definite (compute $\operatorname{tr}(M^2)$ entrywise in each case).
Conclude: $q$ has signature $\bigl(\frac{n(n+1)}{2}, \frac{n(n-1)}{2}\bigr)$ and rank $n^2$ .

Solution

Solution of Exercise 12.12.

$\varphi(M, N) = \operatorname{tr}(MN)$ is bilinear and symmetric ( $\operatorname{tr}(MN) = \operatorname{tr}(NM)$ ), and $\varphi(M, M) = q(M)$ : $q$ is the quadratic form of $\varphi$ .
For $S$ symmetric and $K$ antisymmetric: $\operatorname{tr}(SK) = \operatorname{tr}\bigl((SK)^{\mathsf T}\bigr) = \operatorname{tr}(K^{\mathsf T}S^{\mathsf T}) = -\operatorname{tr}(KS) = -\operatorname{tr}(SK)$ , so $\varphi(S, K) = 0$ : the two subspaces are $\varphi$ -orthogonal. Entrywise, $\operatorname{tr} (M^2) = \sum_{i,j} m_{ij}m_{ji}$ : for symmetric $M$ this is $\sum m_{ij}^2 > 0$ ( $M \neq 0$ ); for antisymmetric $M$ it is $-\sum m_{ij}^2 < 0$ .
$\mathcal M_n(\R) = S_n \oplus A_n$ with dimensions $\frac{n(n+1)}2$ and $\frac{n(n-1)}2$ ; a Gauss reduction adapted to this $\varphi$ -orthogonal splitting writes $q$ as $\frac{n(n+1)}2$ positive and $\frac{n(n-1)}2$ negative squares: signature $\bigl(\frac{n(n+1)}2, \frac{n(n-1)}2\bigr)$ (Sylvester), rank $n^2$ : the form is nondegenerate.

12.5 Problem: Cholesky, Hadamard, and the polar decomposition

Problem 12.1

The spectral theorem is a microscope; this problem uses it as a factory. From Gram matrices we manufacture the Cholesky factorization (and identify the Gauss pivots as ratios of minors), then prove Hadamard’s inequality on determinants, build the polar decomposition $A = QS$ and the singular value decomposition, classify plane conics, and end with the simultaneous reduction of two forms — the theorem behind normal modes of oscillation. Throughout, $E = \R^n$ with its standard inner product.

Part I — Gram matrices and Cholesky. For vectors $v_1, \dots, v_n \in E$ , their Gram matrix is $G = \bigl(\langle v_i, v_j\rangle\bigr)_{i,j}$ .

Show that $G$ is symmetric positive semidefinite, and positive definite if and only if $(v_1, \dots, v_n)$ is linearly independent (compute $X^{\mathsf T}GX$ ).
Conversely, show that every symmetric positive semidefinite $A$ is a Gram matrix: $A = C^{\mathsf T}C$ for some $C$ (use the square root of Exercise 12.6), with $C$ invertible iff $A$ is definite.
Deduce that a positive semidefinite $A$ satisfies $\abs{a_{ij}} \leq \sqrt{a_{ii}\,a_{jj}}$ for all $i, j$ (restrict to two coordinates) — the Cauchy–Schwarz inequality, reread matricially.
(Cholesky) Let $A$ be positive definite. Prove that there is a unique upper triangular $T$ with positive diagonal entries such that

$A = T^{\mathsf T}\,T$

(existence: apply Gram–Schmidt to vectors realizing $A$ as a Gram matrix; uniqueness: if $T_1^{\mathsf T}T_1 = T_2^{\mathsf T}T_2$ , show $T_1T_2^{-1}$ is orthogonal and triangular with positive diagonal, hence $I$ ).
Show that the leading principal minors satisfy $\Delta_k = (t_{11}\cdots t_{kk})^2$ , and deduce that the pivots produced by the Gauss reduction of a positive definite form, taken in the natural variable order, are

$d_k = \frac{\Delta_k}{\Delta_{k-1}} \qquad (\Delta_0 = 1) :$

the minors of Sylvester’s criterion (Exercise 12.8) and the pivots of Gauss are the same data. Check on Example 12.9.

Part II — Hadamard’s inequality.

Let $A$ be positive definite. Prove

$\det A \leq a_{11}\,a_{22}\cdots a_{nn}$

(normalize: $B = DAD$ with $D = \operatorname{diag}(a_{ii}^{-1/2})$ has unit diagonal; bound $\det B = \prod \mu_i$ by AM–GM against $\operatorname{tr} B = n$ ).
Show that equality holds iff $A$ is diagonal.
Deduce Hadamard’s inequality: for every real square matrix $M$ with columns $c_1, \dots, c_n$ ,

$\abs{\det M} \leq \prod_{i=1}^{n}\norm{c_i}_2 ,$

with equality (for invertible $M$ ) iff the columns are pairwise orthogonal (apply questions 6–7 to $M^{\mathsf T}M$ ).
Geometric and combinatorial dividends: interpret question 8 as “the volume of a parallelepiped is at most the product of its edge lengths”; and show that a matrix with all entries in $\intcc{-1}{1}$ has $\abs{\det M} \leq n^{n/2}$ . (Matrices attaining this bound — Hadamard matrices — exist for $n = 1, 2$ and many multiples of $4$ ; whether for all multiples of $4$ is a famous open problem.)

Part III — Polar decomposition and singular values.

Let $A$ be invertible. Show that $A^{\mathsf T}A$ is positive definite, and that

$S = \sqrt{A^{\mathsf T}A} \quad\text{(the square root of \text{Exercise 12.6})}, \qquad Q = AS^{-1}$

give a factorization $A = QS$ with $Q$ orthogonal and $S$ positive definite.
Prove that this factorization of an invertible $A$ is unique.
Extend existence to arbitrary $A$ : choose $\varepsilon_k \to 0$ with $A + \varepsilon_k I$ invertible, write $A + \varepsilon_kI = Q_kS_k$ , and use the compactness of $O(n)$ (Exercise 12.5) to extract $Q_k \to Q$ ; show $S_k = Q_k^{\mathsf T}(A + \varepsilon_kI)$ converges to some positive semidefinite $S$ with $A = QS$ and $S = \sqrt{A^{\mathsf T}A}$ . Where does uniqueness fail for singular $A$ ?
(Singular value decomposition) Deduce that every real square $A$ writes

$A = U\,\Sigma\,V^{\mathsf T}, \qquad U, V \in O(n),\quad \Sigma = \operatorname{diag}(\sigma_1, \dots, \sigma_n),\ \sigma_i \geq 0 ,$

where the $\sigma_i$ (the singular values) are the eigenvalues of $\sqrt{A^{\mathsf T}A}$ .
Three consequences: $\vertiii{A}_2 = \sigma_{\max}$ for every real $A$ (generalizing Exercise 12.7); $\abs{\det A} = \sigma_1\cdots\sigma_n$ ; and the image of the unit sphere under an invertible $A$ is an ellipsoid with semi-axes $\sigma_1, \dots, \sigma_n$ along the columns of $U$ .

Part IV — Conics, by the spectral theorem. A plane conic is the zero set of $f(x) = q(x) + \langle b, x\rangle + c$ , with $q \neq 0$ a quadratic form of matrix $A$ , $b \in \R^2$ , $c \in \R$ .

Reduce $f$ by a rotation (principal axes, Corollary 12.15) followed by a translation, and classify the possible nonempty, nondegenerate shapes by the signature of $q$ : ellipse ( $\det A > 0$ ), hyperbola ( $\det A < 0$ ), parabola ( $\det A = 0$ , rank $1$ , with the linear term not absorbed).
Run the reduction in full for

$x^2 + 4xy + y^2 + 2x - 2y = 4 :$

rotated coordinates, reduced equation, nature and center of the conic.
(Central conics) Suppose $\det A \neq 0$ . Show the center is $x_0 = -\frac12 A^{-1}b$ , and that the congruence by $\begin{pmatrix} I & x_0\\ 0 & 1\end{pmatrix}$ of the $3\times3$ matrix $\widetilde Q = \begin{pmatrix} A & b/2 \\ b^{\mathsf T}/2 & c\end{pmatrix}$ yields

$\det\widetilde Q = f(x_0)\,\det A :$

the central conic is degenerate (a point or two lines) exactly when $\det\widetilde Q = 0$ .
Verify question 17 on the example of question 16: compute $x_0$ , $f(x_0)$ and $\det\widetilde Q$ , and conclude again that the conic is a nondegenerate hyperbola.
(A quadric pencil) For $\lambda \in \R$ , classify the surface

$x^2 + y^2 + z^2 + 2\lambda(xy + yz + zx) = 1$

by the eigenvalues of its matrix (all-ones structure: eigenvalues $1 + 2\lambda$ and $1 - \lambda$ double): sphere/ellipsoid, cylinder, pair of planes, hyperboloids of one and two sheets, according to $\lambda$ .

Part V — Two forms at once: simultaneous reduction.

Let $q$ be positive definite and $q'$ an arbitrary quadratic form on $E$ . Prove that there is a basis of $E$ which is orthonormal for $q$ and orthogonal for $q'$ : in it, $q = \sum x_i^2$ and $q' = \sum \mu_i x_i^2$ (use $q$ as the inner product and apply the spectral theorem to the endomorphism representing $q'$ ).
Matrix form: for $A$ positive definite and $B$ symmetric, there is an invertible $P$ with $P^{\mathsf T}AP = I$ and $P^{\mathsf T}BP = \operatorname{diag}(\mu_1, \dots, \mu_n)$ , where the $\mu_i$ are the roots of $\det(B - \mu A) = 0$ .
Run it in full for

$A = \begin{pmatrix} 2 & 1\\ 1 & 1 \end{pmatrix}, \qquad B = \begin{pmatrix} 0 & 1\\ 1 & 0 \end{pmatrix} :$

the generalized eigenvalues $\mu_\pm$ , and vectors diagonalizing both forms.
Show that positive definiteness cannot be dropped: for

$A = \begin{pmatrix} 1 & 0\\ 0 & -1 \end{pmatrix}, \qquad B = \begin{pmatrix} 0 & 1\\ 1 & 0 \end{pmatrix},$

no basis diagonalizes both forms (if $P$ diagonalized both, $\det(B - \mu A)$ would split with real roots; compute it).
Show that the $\mu_i$ of question 21 are the eigenvalues of $A^{-1}B$ , and that $A^{-1}B$ , while not symmetric in general, is always diagonalizable with real eigenvalues (conjugate by $\sqrt A$ ).
Synthesis. In one sentence each: (i) the one theorem every Part leaned on; (ii) which results of Parts I–III survive for positive semidefinite matrices, and which need definiteness; (iii) the physical system whose small oscillations questions 20–22 diagonalize (kinetic and potential energy as the two forms), and what the $\mu_i$ mean there.

Solution

Solution of Problem 12.1.

1. $G$ is symmetric by symmetry of the inner product, and

X^{\mathsf T}GX = \sum_{i,j}x_ix_j\langle v_i, v_j\rangle = \Bigl\|\sum_i x_iv_i\Bigr\|^2 \geq 0 ,

with equality iff $\sum x_iv_i = 0$ : $G$ is definite iff the only null combination is trivial, i.e. iff the family is independent.

2. With $B = \sqrt A$ (Exercise 12.6): $A = B^2 = B^{\mathsf T}B$ , the Gram matrix of the columns of $B$ ; take $C = B$ . And $X^{\mathsf T}AX = \norm{CX}^2$ , so $A$ is definite iff $CX \neq 0$ for $X \neq 0$ , iff $C$ is invertible.

3. The restriction of the form to $\operatorname{Vect}(e_i, e_j)$ has matrix $\begin{pmatrix} a_{ii} & a_{ij}\\ a_{ij} & a_{jj} \end{pmatrix}$ , still positive semidefinite: its determinant (product of its nonnegative eigenvalues) is $\geq 0$ : $a_{ij}^2 \leq a_{ii}a_{jj}$ . This is Cauchy–Schwarz for the vectors $v_i, v_j$ of a Gram realization.

4. Existence: write $A$ as the Gram matrix of an independent family $(v_1, \dots, v_n)$ (questions 1–2). Gram–Schmidt produces an orthonormal $(e_1, \dots, e_n)$ with

v_k = \sum_{i \leq k} t_{ik}\,e_i, \qquad t_{kk} = \bigl\| v_k - \operatorname{proj}_{k-1}v_k \bigr\| > 0 ,

so $T = (t_{ik})$ is upper triangular with positive diagonal, and

a_{jk} = \langle v_j, v_k\rangle = \sum_i t_{ij}t_{ik} = (T^{\mathsf T}T)_{jk} .

Uniqueness: if $T_1^{\mathsf T}T_1 = T_2^{\mathsf T}T_2$ then $R = T_1T_2^{-1}$ satisfies $R^{\mathsf T}R = I$ : $R$ is orthogonal, and also upper triangular with positive diagonal (product of such). Then $R^{-1} = R^{\mathsf T}$ is simultaneously upper (inverse of upper) and lower (transpose of upper) triangular: diagonal; an orthogonal diagonal matrix has entries $\pm1$ , and positivity forces $R = I$ : $T_1 = T_2$ .

5. For $i, j \leq k$ , $(T^{\mathsf T}T)_{ij} = \sum_m t_{mi}t_{mj}$ only involves $m \leq \min(i,j) \leq k$ : the leading $k\times k$ block of $A$ is $T_k^{\mathsf T}T_k$ with $T_k$ the leading block of $T$ . Hence $\Delta_k = (\det T_k)^2 = (t_{11}\cdots t_{kk})^2$ . Now the Gauss reduction of a positive definite form in the natural order never meets a zero square coefficient (the pivots are the diagonal entries of the successively reduced positive definite blocks): it produces $q = \sum_k d_k\ell_k^2$ with $\ell_k = x_k + (\text{terms in } x_{k+1}, \dots)$ , i.e. $A = L^{\mathsf T}DL$ with $L$ unipotent triangular; then $T = \sqrt D\,L$ is a Cholesky factor, so by uniqueness $t_{kk}^2 = d_k$ and

d_k = \frac{(t_{11}\cdots t_{kk})^2} {(t_{11}\cdots t_{k-1,k-1})^2} = \frac{\Delta_k}{\Delta_{k-1}} .

On Example 12.9: $\Delta_1, \Delta_2, \Delta_3 = 2, 3, 4$ and the pivots were $2, \frac32, \frac43$ .

6. Each $a_{ii} = e_i^{\mathsf T}Ae_i > 0$ . Let $D = \operatorname{diag}(a_{ii}^{-1/2})$ and $B = DAD$ : positive definite (congruence), with $b_{ii} = 1$ , so $\operatorname{tr} B = n$ . Its eigenvalues $\mu_i > 0$ satisfy, by AM–GM,

\det B = \prod_i\mu_i \leq \Bigl(\frac{\sum\mu_i}{n}\Bigr)^{\!n} = 1 ,

and $\det B = (\det D)^2\det A = \dfrac{\det A}{\prod a_{ii}}$ : $\det A \leq \prod a_{ii}$ .

7. AM–GM is an equality iff all $\mu_i$ are equal (to $1$ ); a symmetric matrix with sole eigenvalue $1$ is $PIP^{\mathsf T} = I$ . So equality iff $B = I$ iff $a_{ij} = 0$ for $i \neq j$ : $A$ diagonal.

8. If $M$ is singular both sides are $\geq 0 = \abs{\det M}$ . Otherwise $A = M^{\mathsf T}M$ is positive definite with $a_{ii} = \norm{c_i}^2$ and $\det A = (\det M)^2$ : question 6 gives $(\det M)^2 \leq \prod\norm{c_i}^2$ . Equality iff $A = M^{\mathsf T}M$ is diagonal (question 7), i.e. iff the columns are pairwise orthogonal.

9. $\abs{\det M}$ is the volume of the parallelepiped spanned by the columns: the volume is at most the product of the edge lengths, with equality exactly for rectangular boxes. If $\abs{m_{ij}} \leq 1$ then $\norm{c_i} \leq \sqrt n$ , so $\abs{\det M} \leq n^{n/2}$ . (Attaining it forces orthogonal columns of entries $\pm1$ : a Hadamard matrix.)

10. $X^{\mathsf T}A^{\mathsf T}AX = \norm{AX}^2 > 0$ for $X \neq 0$ ( $A$ invertible): $A^{\mathsf T}A$ is positive definite. Its square root $S$ is positive definite (eigenvalues $\sqrt{\lambda_i} > 0$ ), hence invertible, and $Q = AS^{-1}$ satisfies

Q^{\mathsf T}Q = S^{-1}A^{\mathsf T}AS^{-1} = S^{-1}S^2S^{-1} = I :

$A = QS$ with $Q$ orthogonal, $S$ positive definite.

11. If $A = QS = Q'S'$ then $S'^{\,2} = S'^{\mathsf T}Q'^{\mathsf T}Q'S' = A^{\mathsf T}A = S^2$ ; two positive semidefinite matrices with the same square coincide (Exercise 12.6): $S' = S$ , then $Q' = AS^{-1} = Q$ .

12. $\det(A + \varepsilon I)$ is a nonzero polynomial in $\varepsilon$ : it has finitely many roots, so some sequence $\varepsilon_k \to 0$ avoids them. Write $A + \varepsilon_kI = Q_kS_k$ (question 10). $O(n)$ is compact (Exercise 12.5): a subsequence gives $Q_{\varphi(k)} \to Q \in O(n)$ . Then

S_{\varphi(k)} = Q_{\varphi(k)}^{\mathsf T} \bigl(A + \varepsilon_{\varphi(k)}I\bigr) \longrightarrow Q^{\mathsf T}A =: S,

symmetric positive semidefinite as a limit of such (closed conditions), and $A = QS$ . Moreover $S^2 = S^{\mathsf T}S = A^{\mathsf T}QQ^{\mathsf T}A = A^{\mathsf T}A$ , so $S = \sqrt{A^{\mathsf T}A}$ by uniqueness. For singular $A$ , $S$ is singular and $Q$ is not unique: it can be modified arbitrarily on $(\operatorname{im} S)^{\perp}$ — extreme case $A = 0$ , where every orthogonal $Q$ works.

13. Diagonalize $S = P\Sigma P^{\mathsf T}$ (spectral theorem), $\Sigma = \operatorname{diag}(\sigma_i)$ with $\sigma_i \geq 0$ the eigenvalues of $S = \sqrt{A^{\mathsf T}A}$ . Then

A = QS = (QP)\,\Sigma\,P^{\mathsf T} = U\Sigma V^{\mathsf T}, \qquad U = QP,\ V = P \in O(n) .

14. $\norm{Ax}^2 = x^{\mathsf T}S^2x \leq \sigma_{\max}^2\norm x^2$ with equality at a top eigenvector of $S$ : $\vertiii A_2 = \sigma_{\max}$ — for symmetric $A$ , $S = \sqrt{A^2}$ has eigenvalues $\abs{\lambda_i}$ , recovering Exercise 12.7. Determinant: $\abs{\det A} = \abs{\det U}\det\Sigma\abs{\det V} = \sigma_1\cdots\sigma_n$ . Sphere: writing $x = Vy$ with $\norm y = 1$ , $Ax = U\Sigma y$ has coordinates $z_i = \sigma_iy_i$ in the orthonormal frame of the columns of $U$ : the image is $\{\sum z_i^2/\sigma_i^2 = 1\}$ , an ellipsoid with semi-axes $\sigma_i$ .

15. A rotation to principal axes (Corollary 12.15) turns $f$ into $\lambda_1X^2 + \lambda_2Y^2 + \beta_1X + \beta_2Y + c$ , with $\lambda_1\lambda_2 = \det A$ . If $\det A \neq 0$ , absorb the linear terms by the translation $X \mapsto X - \frac{\beta_1}{2\lambda_1}$ (and likewise $Y$ ): $\lambda_1X'^2 + \lambda_2Y'^2 = c'$ . For $\det A > 0$ (equal signs): an ellipse ( $c'$ of the right sign), a point, or empty. For $\det A < 0$ : a hyperbola ( $c' \neq 0$ ) or two crossing lines. If $\det A = 0$ with rank $1$ (say $\lambda_2 = 0 \neq \lambda_1$ ): $\lambda_1X'^2 + \beta_2Y + c''$ , a parabola when $\beta_2 \neq 0$ ; otherwise two parallel lines, one line, or empty. Nondegenerate shapes: ellipse, hyperbola, parabola, governed by the sign of $\det A$ .

16. The quadratic part $x^2 + 4xy + y^2$ has matrix $\begin{pmatrix}1 & 2\\ 2 & 1\end{pmatrix}$ , eigenvalues $3$ and $-1$ with orthonormal directions $\frac{1}{\sqrt2}(1,1)$ , $\frac{1}{\sqrt2}(1,-1)$ . In the rotated coordinates $u = \frac{x+y}{\sqrt2}$ , $v = \frac{x-y}{\sqrt2}$ : $x^2 + y^2 = u^2 + v^2$ , $2xy = u^2 - v^2$ , so the form is $3u^2 - v^2$ , and $2x - 2y = 2\sqrt2\,v$ . The equation becomes

3u^2 - v^2 + 2\sqrt2\,v = 4 \quad\Longleftrightarrow\quad 3u^2 - \bigl(v - \sqrt2\bigr)^2 = 2 :

a hyperbola, center at $(u, v) = (0, \sqrt2)$ , i.e. $(x, y) = (1, -1)$ , with axes along the rotated frame.

17. $f(x) = (x - x_0)^{\mathsf T}A(x - x_0) + f(x_0)$ whenever $Ax_0 = -\frac b2$ , i.e. $x_0 = -\frac12A^{-1}b$ : the gradient of $f$ vanishes exactly there ( $x_0$ is the center of symmetry). With $M = \begin{pmatrix} I & x_0\\ 0 & 1\end{pmatrix}$ :

M^{\mathsf T}\widetilde QM = \begin{pmatrix} A & Ax_0 + \frac b2\\[2pt] \bigl(Ax_0 + \frac b2\bigr)^{\mathsf T} & x_0^{\mathsf T}Ax_0 + b^{\mathsf T}x_0 + c \end{pmatrix} = \begin{pmatrix} A & 0\\ 0 & f(x_0)\end{pmatrix},

and $\det M = 1$ : $\det\widetilde Q = f(x_0)\det A$ . The centered equation reads $q(X) = -f(x_0)$ : for $f(x_0) = 0$ it degenerates to $q(X) = 0$ (two lines through the center if the signature is $(1,1)$ , the single point $x_0$ if $q$ is definite); for $f(x_0) \neq 0$ the conic is a genuine ellipse or hyperbola.

18. $A^{-1} = -\frac13\begin{pmatrix} 1 & -2\\ -2 & 1\end{pmatrix}$ , $\frac b2 = (1, -1)$ : $x_0 = -A^{-1}\frac b2 = (1, -1)$ , as found in question 16. $f(x_0) = q(1,-1) + 2 + 2 - 4 = (1 - 4 + 1) + 0 = -2 \neq 0$ , and $\det\widetilde Q = f(x_0)\det A = (-2)(-3) = 6 \neq 0$ : nondegenerate; $\det A = -3 < 0$ : a hyperbola — and indeed the centered equation $3u^2 - (v - \sqrt2)^2 = -f(x_0) = 2$ matches question 16.

19. The matrix is $(1-\lambda)I + \lambda J$ : eigenvalues $1 + 2\lambda$ (direction $(1,1,1)$ ) and $1 - \lambda$ (double, on $x + y + z = 0$ ). Cases:

$-\frac12 < \lambda < 1$ : all eigenvalues positive: an ellipsoid of revolution about $(1,1,1)$ (a sphere for $\lambda = 0$ );
$\lambda = 1$ : $q = (x+y+z)^2$ : the equation gives the two parallel planes $x + y + z = \pm1$ ;
$\lambda = -\frac12$ : eigenvalues $0, \frac32, \frac32$ : a circular cylinder of axis $(1,1,1)$ ;
$\lambda > 1$ : signature $(1, 2)$ : a hyperboloid of two sheets;
$\lambda < -\frac12$ : signature $(2, 1)$ : a hyperboloid of one sheet.

20. The polar form of $q$ is an inner product $\langle\cdot,\cdot\rangle_q$ on $E$ . For fixed $x$ , $y \mapsto \varphi'(x, y)$ (polar form of $q'$ ) is linear, hence equals $\langle z_x, y\rangle_q$ for a unique $z_x$ ; $u(x) := z_x$ is linear (uniqueness), and $\langle u(x), y\rangle_q = \varphi'(x,y) = \varphi'(y,x) = \langle u(y), x\rangle_q$ : $u$ is symmetric in the Euclidean space $(E, \langle\cdot,\cdot\rangle_q)$ . The spectral theorem (Theorem 12.13) gives a $q$ -orthonormal eigenbasis $(\varepsilon_i)$ , $u(\varepsilon_i) = \mu_i\varepsilon_i$ : in it $q(x) = \sum x_i^2$ and $q'(x) = \langle u(x), x\rangle_q = \sum\mu_ix_i^2$ .

21. Let $P$ be the matrix of that basis: congruence gives $P^{\mathsf T}AP = I$ and $P^{\mathsf T}BP = \operatorname{diag}(\mu_i)$ . Then

\det(B - \mu A) = \det(P^{-\mathsf T}) \det\bigl(\operatorname{diag}(\mu_i) - \mu I\bigr) \det(P^{-1}) = (\det P)^{-2}\prod_i(\mu_i - \mu) :

the $\mu_i$ are the roots of the pencil $\det(B - \mu A)$ .

22. $\det(B - \mu A) = \det\begin{pmatrix} -2\mu & 1-\mu\\ 1-\mu & -\mu\end{pmatrix} = 2\mu^2 - (1-\mu)^2 = \mu^2 + 2\mu - 1$ : roots $\mu_\pm = -1 \pm \sqrt2$ . Solving $(B - \mu_\pm A)v = 0$ : $v_\pm = (1 - \mu_\pm,\ 2\mu_\pm)$ (the second row identity $(1-\mu)^2 = 2\mu^2$ at the roots confirms it). The $A$ -norms come out clean: $q_A(v_\pm) = 2(1 + \mu_\pm^2)$ , and one checks $\varphi_A(v_+, v_-) = 0$ using $\mu_+ + \mu_- = -2$ , $\mu_+\mu_- = -1$ . The basis $\Bigl(\frac{v_+}{\sqrt{2(1 + \mu_+^2)}}, \frac{v_-}{\sqrt{2(1+\mu_-^2)}}\Bigr)$ is orthonormal for $A$ and diagonalizes $B$ with entries $\mu_\pm$ .

23. If some invertible $P$ diagonalized both forms, question 21’s computation would give $\det(B - \mu A) = (\det P)^{-2}\prod(d_{2i} - \mu d_{1i})$ , a real polynomial split into real linear factors. But here

\det(B - \mu A) = \det\begin{pmatrix} -\mu & 1\\ 1 & \mu\end{pmatrix} = -\mu^2 - 1 ,

of degree $2$ with no real root: contradiction. (The Lorentz-signature $A$ admits $B$ -“rotations” with no real axes.)

24. $A^{-1}B = A^{-1/2}\bigl(A^{-1/2}BA^{-1/2}\bigr) A^{1/2}$ with $A^{1/2} = \sqrt A$ positive definite (Exercise 12.6): $A^{-1}B$ is similar to the symmetric $A^{-1/2}BA^{-1/2}$ , hence diagonalizable with real eigenvalues. And $\det(B - \mu A) = \det A\cdot \det(A^{-1}B - \mu I)$ : the pencil roots $\mu_i$ of question 21 are exactly the eigenvalues of $A^{-1}B$ .

25. (i) Every Part leaned on the spectral theorem: through the square root (Cholesky, polar), the eigenvalue bounds (Hadamard), the principal axes (conics), and the $q$ -adapted version (simultaneous reduction). (ii) Gram realizations, Hadamard, and the polar decomposition survive in the semidefinite world; Cholesky’s uniqueness, the pivot formula, and the simultaneous reduction need definiteness (questions 12 and 23 show exactly how they fail). (iii) Coupled small oscillations: kinetic energy (positive definite) and potential energy are two quadratic forms; the basis of questions 20–22 is the system’s normal modes, and the $\mu_i$ are the squared angular frequencies.