University Mathematics — Year 2 · Bachelor Year 2

13Hermitian Forms

Complex vector spaces have their own inner-product geometry, with one twist: linearity in one variable, conjugate-linearity in the other. The payoff for accepting the twist is a spectral theory even cleaner than the real one — Hermitian endomorphisms have real eigenvalues, unitary ones have unimodular eigenvalues, and both diagonalize in orthonormal bases. This short chapter runs the Euclidean program of Chapter 12 over $\C$ .

13.1 Hermitian inner products

Definition 13.1

A Hermitian inner product on a complex vector space $E$ is a map $\langle\cdot,\cdot\rangle \colon E \times E \to \C$ that is linear in the second variable, conjugate-symmetric ( $\langle y, x\rangle = \conj{\langle x, y\rangle}$ — hence conjugate-linear in the first variable), and positive definite ( $\langle x, x\rangle > 0$ for $x \neq 0$ ). The standard example on $\C^n$ :

\langle x, y \rangle = \sum_{i=1}^{n} \conj{x_i}\, y_i ;

on continuous functions, $\langle f, g\rangle = \int_a^b \conj f\,g$ . Norm: $\norm x = \sqrt{\langle x,x\rangle}$ ; a finite-dimensional complex space so equipped is a Hermitian space.

Example 13.2 (First computations)

In $\C^2$ , take $x = (1+\iu,\ 2-\iu)$ and $y = (\iu,\ 1)$ . Then

\norm x^2 = \abs{1+\iu}^2 + \abs{2-\iu}^2 = 2 + 5 = 7, \qquad \norm y^2 = 1 + 1 = 2,

and, conjugating the first argument,

\langle x, y\rangle = \conj{(1+\iu)}\,\iu + \conj{(2-\iu)}\cdot1 = (1-\iu)\iu + (2+\iu) = (\iu + 1) + (2 + \iu) = 3 + 2\iu .

Cauchy–Schwarz checks out: $\abs{\langle x, y\rangle}^2 = 9 + 4 = 13 \leq 14 = \norm x^2\norm y^2$ — close to equality, because $x$ is close to a multiple of $y$ . Note also $\langle y, x\rangle = \conj{3 + 2\iu} = 3 - 2\iu$ : conjugate-symmetry in action, and the reason $\langle x, x\rangle$ is always real.

Theorem 13.3 (Cauchy–Schwarz, complex)

$\abs{\langle x, y\rangle} \leq \norm x \norm y$ , with equality iff $x, y$ are linearly dependent; $\norm\cdot$ is a norm. Moreover orthonormal bases exist (Gram–Schmidt runs verbatim), with

x = \sum_i \langle e_i, x\rangle\, e_i, \qquad \norm x^2 = \sum_i \abs{\langle e_i, x\rangle}^2 .

Proof. For $y \neq 0$ and $t \in \C$ : $0 \leq \norm{x - ty}^2 = \norm x^2 - 2\Re\bigl(\conj t\langle y, x\rangle\bigr) + \abs t^2\norm y^2$ . Choose $t = \frac{\langle y, x\rangle}{\norm y^2}$ :

0 \leq \norm x^2 - \frac{\abs{\langle y, x\rangle}^2}{\norm y^2},

which is the inequality; equality forces $x = ty$ . Triangle inequality, in full:

\norm{x + y}^2 = \norm x^2 + 2\,\Re\langle x, y\rangle + \norm y^2 \leq \norm x^2 + 2\,\abs{\langle x, y\rangle} + \norm y^2 \leq \bigl(\norm x + \norm y\bigr)^2 ,

using $\Re z \leq \abs z$ and then Cauchy–Schwarz; homogeneity and separation are immediate, so $\norm\cdot$ is a norm. Gram–Schmidt: as in the real case, with conjugates placed by the definition (note the convention: our products are conjugate-linear in the first slot, so coordinates are $\langle e_i, x\rangle$ ; the next example runs the algorithm once in full). ∎

Example 13.4 (Complex Gram–Schmidt, run in full)

Orthonormalize the basis $v_1 = (1, \iu)$ , $v_2 = (0, 1)$ of $\C^2$ . First vector: $\norm{v_1}^2 = \abs1^2 + \abs\iu^2 = 2$ , so $e_1 = \frac{1}{\sqrt2}(1, \iu)$ . Project $v_2$ — with the conjugate in the first slot:

\langle e_1, v_2\rangle = \frac{1}{\sqrt2}\bigl(\conj{1}\cdot0 + \conj{\iu}\cdot1\bigr) = \frac{-\iu}{\sqrt2} , \qquad v_2 - \langle e_1, v_2\rangle e_1 = (0,1) + \frac{\iu}{2}\,(1, \iu) = \Bigl(\frac\iu2,\ \frac12\Bigr) .

Its norm is $\sqrt{\frac14 + \frac14} = \frac{1}{\sqrt2}$ : $e_2 = \frac{1}{\sqrt2}(\iu, 1)$ . Check: $\langle e_1, e_2\rangle = \frac12(\conj1\cdot\iu + \conj\iu\cdot1) = \frac12(\iu - \iu) = 0$ . Coordinates of $v_2$ in the new basis: $v_2 = \langle e_1, v_2\rangle e_1 + \langle e_2, v_2\rangle e_2$ with $\langle e_2, v_2\rangle = \frac{1}{\sqrt2}$ — mind the order: $\langle v_2, e_1\rangle$ would give the conjugate coefficient. Closing insight: the algorithm is the Euclidean one verbatim; the only trap is where the conjugation falls, and computing $\norm{v_2 - \text{proj}}^2 > 0$ silently uses positivity — the axiom that makes the whole geometry work.

13.2 Adjoint, Hermitian and unitary endomorphisms

Definition 13.5

The adjoint $u^*$ of $u \in \mathcal{L}(E)$ is defined by $\langle u^*(x), y\rangle = \langle x, u(y)\rangle$ ; in an orthonormal basis, $\operatorname{Mat}(u^*) = \conj{A}^{\mathsf T} =: A^{\dagger}$ (conjugate transpose) — indeed, if $B = (b_{ij})$ is the matrix of $u^*$ in the orthonormal basis $(e_i)$ , then $b_{ij} = \langle e_i, u^*(e_j)\rangle$ , and the defining identity gives

\conj{b_{ij}} = \langle u^*(e_j), e_i\rangle = \langle e_j, u(e_i)\rangle = a_{ji} , \qquad\text{so}\qquad B = \conj{A}^{\mathsf T} .

$u$ is Hermitian when $u^* = u$ ( $A^\dagger = A$ ), unitary when $u^*u = \mathrm{id}$ ( $A^\dagger A = I$ : the group $U(n)$ ), normal when $u^*u = uu^*$ .

Proposition 13.6

Eigenvalues of a Hermitian endomorphism are real; eigenvalues of a unitary endomorphism have modulus $1$ ; in both cases eigenspaces for distinct eigenvalues are orthogonal.

Proof. Hermitian, $u(x) = \lambda x$ , $x \neq 0$ :

\lambda \norm x^2 = \langle x, u(x)\rangle = \langle u(x), x\rangle = \conj{\langle x, u(x)\rangle} = \conj\lambda\,\norm x^2 ,

so $\lambda \in \R$ . Unitary: $\norm{u(x)} = \norm x$ (from $u^*u = \mathrm{id}$ ), so $\abs\lambda\norm x = \norm x$ . Orthogonality (Hermitian case): $\lambda\langle x, y\rangle = \langle u(x), y\rangle = \langle x, u(y)\rangle = \mu\langle x, y\rangle$ for eigenvectors with real $\lambda \neq \mu$ . Unitary case, in full: for $u(x) = \lambda x$ , $u(y) = \mu y$ with $\lambda \neq \mu$ (both unimodular),

\langle x, y\rangle = \langle u(x), u(y)\rangle = \conj\lambda\mu\,\langle x, y\rangle ,

and $\conj\lambda\mu = \frac{\mu}{\lambda} \neq 1$ : the factor is not $1$ , so $\langle x, y\rangle = 0$ . ∎

Example 13.7 (A skew-Hermitian matrix, diagonalized)

$A = \begin{pmatrix} 0 & -2\\ 2 & 0\end{pmatrix}$ satisfies $A^\dagger = A^{\mathsf T} = -A$ : skew-Hermitian (also real antisymmetric — over $\R$ it has no eigenvalues at all). Characteristic polynomial $X^2 + 4$ : eigenvalues $\pm2\iu$ , purely imaginary, as Exercise 13.9 predicts in general. Eigenvectors: $(A - 2\iu I)v = 0$ gives $v_1 = \frac{1}{\sqrt2}(1, \iu)$ , and $v_2 = \frac{1}{\sqrt2}(1, -\iu)$ for $-2\iu$ ; they are orthogonal:

\langle v_1, v_2\rangle = \tfrac12\bigl(\conj{1}\cdot1 + \conj{\iu}\cdot(-\iu)\bigr) = \tfrac12(1 - 1) = 0 .

So $A = U\operatorname{diag}(2\iu, -2\iu)\,U^\dagger$ with $U = (v_1\ v_2)$ unitary. Closing insight: $H = -\iu A = \begin{pmatrix} 0 & 2\iu\\ -2\iu & 0\end{pmatrix}$ is Hermitian with the real spectrum $\{\pm2\}$ and the same eigenvectors — the bijection $u \mapsto \iu u$ between Hermitian and skew-Hermitian endomorphisms (Exercise 13.9), seen matrix by matrix; over $\R$ the same $A$ is a rotation-scaling with no eigenvectors at all, and only the passage to $\C$ reveals its normal form.

Theorem 13.8 (Hermitian spectral theorem)

Every Hermitian endomorphism of a Hermitian space has an orthonormal basis of eigenvectors (with real eigenvalues): $A^\dagger = A$ implies $A = U D U^{\dagger}$ with $U \in U(n)$ and $D$ real diagonal.

Proof. Over $\C$ , the characteristic polynomial splits: an eigenvector $e_1$ exists (Chapter 3) — no compactness argument needed, one advantage of $\C$ . Normalize it. Its orthogonal complement $F = e_1^\perp$ is stable: for $x \perp e_1$ ,

\langle e_1, u(x)\rangle = \langle u(e_1), x\rangle = \lambda_1\langle e_1, x\rangle = 0

( $\lambda_1$ real). The restriction is Hermitian; induct on the dimension and concatenate. In detail: the restriction $u|_F$ is an endomorphism of the Hermitian space $F$ (dimension $n - 1$ ) with $\langle u|_F(x), y\rangle = \langle x, u|_F(y)\rangle$ inherited from $u$ ; the induction hypothesis provides an orthonormal basis $(e_2, \dots, e_n)$ of $F$ of eigenvectors, and $(e_1, e_2, \dots, e_n)$ is orthonormal in $E$ ( $e_1 \perp F$ ) and made of eigenvectors of $u$ . Matrix translation: the columns of $U$ are the $e_i$ , $U^\dagger U = I$ expresses their orthonormality, and $AU = UD$ collects the eigenvalue equations, whence $A = UDU^\dagger$ with $D$ real diagonal (Proposition 13.6). ∎

Example 13.9

$A = \begin{pmatrix} 0 & -\iu\\ \iu & 0\end{pmatrix}$ is Hermitian ( $A^\dagger = A$ ): eigenvalues from $\chi_A = X^2 - 1$ : $\pm 1$ (real, as promised), with orthonormal eigenvectors $\frac{1}{\sqrt2}(1, \iu)^{\mathsf T}$ and $\frac{1}{\sqrt2}(1, -\iu)^{\mathsf T}$ . (Physicists know $A$ as a Pauli matrix; the reality of Hermitian spectra is why quantum observables are modeled by Hermitian operators.)

Example 13.10 (A positive definite Hermitian matrix, worked)

$A = \begin{pmatrix} 2 & 1-\iu\\ 1+\iu & 3\end{pmatrix}$ : Hermitian, since the diagonal is real and the off-diagonal entries are conjugates. Characteristic polynomial:

(2-\lambda)(3-\lambda) - \abs{1-\iu}^2 = \lambda^2 - 5\lambda + 4 = (\lambda - 1)(\lambda - 4) :

spectrum $\{1, 4\}$ , real and positive — $A$ is positive definite. Eigenvectors: for $\lambda = 4$ , the system $(A - 4I)v = 0$ gives $v_4 = (1 - \iu,\ 2)$ (check the second row: $(1+\iu)(1-\iu) - 2 = 0$ ); for $\lambda = 1$ , $v_1 = (1 - \iu,\ -1)$ . Orthogonality, with the conjugate in the first slot:

\langle v_4, v_1\rangle = \conj{(1-\iu)}\,(1-\iu) + \conj{2}\,(-1) = 2 - 2 = 0 . \checkmark

Normalizing ( $\norm{v_4}^2 = 2 + 4 = 6$ , $\norm{v_1}^2 = 2 + 1 = 3$ ) gives the unitary $U = \bigl(\frac{v_4}{\sqrt6}\ \frac{v_1}{\sqrt3}\bigr)$ with $A = U\operatorname{diag}(4,1)U^\dagger$ . Closing insight: the Rayleigh reading is immediate — on the unit sphere of $\C^2$ , $\langle x, Ax\rangle$ ranges over $\intcc{1}{4}$ , attained at the two eigenvectors; this is the $n = 2$ germ of the Courant–Fischer theory built in the weekend problem. Spot-check that the form is real off the eigenvectors too: at $x = (1, \iu)$ ,

Ax = \bigl(2 + (1-\iu)\iu,\ (1+\iu) + 3\iu\bigr) = (3 + \iu,\ 1 + 4\iu),

\langle x, Ax\rangle = \conj{1}\,(3+\iu) + \conj{\iu}\,(1+4\iu) = (3 + \iu) + (-\iu)(1 + 4\iu) = 3 + \iu - \iu + 4 = 7 \in \R ,

as Proposition 13.6’s proof mechanism (conjugate-symmetry against $A^\dagger = A$ ) guarantees for every $x$ .

Example 13.11 (A unitary matrix diagonalized)

$U = \frac{1}{\sqrt2}\begin{pmatrix} 1 & \iu\\ \iu & 1 \end{pmatrix}$ (unitary by Exercise 13.2). Its characteristic polynomial is $\bigl(X - \frac{1}{\sqrt2}\bigr)^2 + \frac12$ , with roots

\lambda_\pm = \frac{1 \pm \iu}{\sqrt2} = \eu^{\pm\iu\pi/4},

unimodular as Proposition 13.6 promised, and orthonormal eigenvectors $\frac{1}{\sqrt2}(1, \pm1)$ . So $U = V\operatorname{diag}(\eu^{\iu\pi/4}, \eu^{-\iu\pi/4})V^\dagger$ : in the right basis, $U$ is a pair of plane rotations by $\pm\frac\pi4$ — a real rotation matrix has no real eigenvectors, but over $\C$ it splits into two unimodular scalars. Closing insight: Hermitian spectra live on the real line, unitary spectra on the unit circle; both are shadows of the same normality, and the Cayley transform of Exercise 13.6 maps one picture to the other.

Example 13.12 (The Cayley transform, computed)

Run Exercise 13.6 on $H = \begin{pmatrix} 0 & 1\\ 1 & 0\end{pmatrix}$ (Hermitian, spectrum $\{1, -1\}$ , orthonormal eigenvectors $\frac{1}{\sqrt2}(1, \pm1)$ ). In the eigenbasis everything is scalar: the transform $\lambda \mapsto \frac{\lambda - \iu}{\lambda + \iu}$ sends

1 \longmapsto \frac{1 - \iu}{1 + \iu} = -\iu, \qquad -1 \longmapsto \frac{-1 - \iu}{-1 + \iu} = \iu ,

(multiply by the conjugate of the denominator), so $U = (H - \iu I)(H + \iu I)^{-1}$ is the unitary with eigenvalues $\mp\iu$ on those same eigenvectors:

U = \frac{1}{2}\begin{pmatrix} 1 & 1\\ 1 & -1\end{pmatrix} \begin{pmatrix} -\iu & 0\\ 0 & \iu\end{pmatrix} \begin{pmatrix} 1 & 1\\ 1 & -1\end{pmatrix} = \begin{pmatrix} 0 & -\iu\\ -\iu & 0\end{pmatrix} .

Check: $U^\dagger U = I$ , and $1 \notin \operatorname{Sp}U = \{\pm\iu\}$ , as the theory promises. Closing insight: the real line maps onto the unit circle minus the point $1$ — eigenvalue by eigenvalue, the Cayley transform is the Möbius map $\frac{\lambda-\iu}{\lambda+\iu}$ , and matrices just follow their spectra.

Remark 13.13 (Common pitfalls)

(i) Where the bar falls: this book conjugates the first slot, so coordinates are $\langle e_i, x\rangle$ and $\langle\lambda x, y\rangle = \conj\lambda\langle x, y\rangle$ ; many texts conjugate the second slot instead — translate before comparing formulas, or signs of $\iu$ go wrong silently. (ii) Complex polarization is stronger: over $\C$ , if $\langle x, u(x)\rangle = 0$ for all $x$ then $u = 0$ (expand $x + y$ and $x + \iu y$ : both the real and imaginary parts of $\langle x, u(y)\rangle$ vanish); over $\R$ this fails — the rotation by $\frac\pi2$ satisfies $\langle x, u(x)\rangle = 0$ everywhere. Consequently, over $\C$ only, “ $\langle x, u(x)\rangle \in \R$ for all $x$ ” already forces $u$ Hermitian. (iii) Real normal is not diagonalizable: the matrix of Example 13.7 is normal but has no real eigenvalue; unitary diagonalization is a theorem over $\C$ , and over $\R$ one only gets block reductions. (iv) Checking unitarity: $U^\dagger U = I$ means the columns are orthonormal for the Hermitian product — testing $UU^{\mathsf T}$ , or forgetting the conjugation in the column products, are the two classic ways to certify a wrong matrix.

Example 13.14 (Isometries are exactly the unitaries)

Norm preservation looks weaker than unitarity, but over $\C$ it is not: if $\norm{u(x)} = \norm x$ for all $x$ , then $u^*u = \mathrm{id}$ . Indeed $v = u^*u - \mathrm{id}$ is Hermitian and satisfies $\langle x, v(x)\rangle = \norm{u(x)}^2 - \norm x^2 = 0$ for every $x$ ; by complex polarization (pitfall (ii) above), a map with identically vanishing “diagonal” is zero: $v = 0$ . Concretely, the polarization runs

0 = \langle x + y, v(x+y)\rangle = \langle x, v(y)\rangle + \langle y, v(x)\rangle, \qquad 0 = \langle x + \iu y, v(x + \iu y)\rangle = \iu\langle x, v(y)\rangle - \iu\langle y, v(x)\rangle ,

and the two lines together force $\langle x, v(y)\rangle = 0$ for all $x, y$ . Closing insight: this is why “unitary” can be checked by measuring lengths alone — rigidity that the Fourier chapter will exploit, where preserving the energy $\norm f_2$ (Parseval) is the same as preserving all inner products of coefficients.

Remark 13.15 (Perspectives within this volume)

The Hermitian machinery built here is consumed almost immediately. The Fourier chapter is Hermitian geometry in infinite dimension: the exponentials $(e_n)$ are an orthonormal family for $\langle f, g\rangle = \frac{1}{2\pi}\int\conj fg$ , Bessel’s inequality is the projection estimate of this chapter’s Theorem 13.3, and Parseval is its limiting equality. The finite Fourier transform (Exercise 13.10) reappears whenever convolution must be diagonalized. And this chapter’s weekend problem — Courant–Fischer, Weyl, interlacing — supplies the eigenvalue stability that the differential-equations chapter invokes when it asserts that small perturbations of a system move its frequencies only slightly. Backward, everything here is the complex mirror of the quadratic-forms chapter: keep the two dictionaries side by side ( $A^{\mathsf T} \leftrightarrow A^\dagger$ , orthogonal $\leftrightarrow$ unitary, Rayleigh real in both).

Remark 13.16 (Normal endomorphisms)

Over $\C$ the definitive statement is: $u$ is unitarily diagonalizable if and only if it is normal ( $u^*u = uu^*$ ) — covering Hermitian, unitary and skew-Hermitian maps at once. The proof is a pleasant strengthening of the argument above (Exercise 13.8). Over $\R$ , by contrast, normality only buys block-diagonalization (rotation blocks): complex geometry is genuinely simpler.

Remark 13.17 (Where this is used)

Hermitian spectral theory is the mathematics of quantum mechanics: observables are modeled by Hermitian operators (real spectra = measurable values), time evolution by unitary ones (norm preservation = conservation of probability). Within this book, the Fourier chapter rests on the orthonormality of the exponentials — a Hermitian inner-product statement — and the diagonalization of circulant matrices (Exercise 13.10) is the finite Fourier transform. The weekend problem develops the variational calculus of eigenvalues (Courant–Fischer, Weyl, interlacing), the daily bread of numerical analysis and mathematical physics; the Year 3 volume extends it to compact self-adjoint operators on Hilbert spaces.

13.3 Exercises

Exercise 13.1 ★

On $\C^2$ : compute $\langle x, y\rangle$ , $\norm x$ , $\norm y$ for $x = (1, \iu)$ , $y = (\iu, 1)$ ; are they orthogonal? Give an orthonormal basis containing $\frac{x}{\norm x}$ .

Solution

Solution of Exercise 13.1.

$\langle x, y\rangle = \conj{1}\cdot\iu + \conj{\iu}\cdot 1 = \iu - \iu = 0$ : orthogonal. $\norm x = \norm y = \sqrt{1 + 1} = \sqrt2$ . Orthonormal basis: $\bigl(\frac{1}{\sqrt2}(1, \iu),\; \frac{1}{\sqrt2}(\iu, 1)\bigr)$ — the normalized pair itself.

Exercise 13.2 ★

Which are Hermitian? unitary? normal?

\begin{pmatrix} 1 & \iu\\ -\iu & 2 \end{pmatrix}, \qquad \frac{1}{\sqrt2}\begin{pmatrix} 1 & \iu\\ \iu & 1\end{pmatrix}, \qquad \begin{pmatrix} 0 & 1\\ 0 & 0 \end{pmatrix}.

Solution

Solution of Exercise 13.2.

First: equals its conjugate transpose (real diagonal, $\conj{\iu} = -\iu$ swapped): Hermitian (hence normal); not unitary ( $A^\dagger A \neq I$ : columns not unit).

Second: $A^\dagger A = \frac12\begin{pmatrix} 1 & -\iu\\ -\iu & 1\end{pmatrix}\begin{pmatrix} 1 & \iu\\ \iu & 1\end{pmatrix} = \frac12\begin{pmatrix} 2 & 0\\ 0 & 2\end{pmatrix} = I$ : unitary (hence normal); not Hermitian.

Third: $A^\dagger A = E_{22} \neq E_{11} = AA^\dagger$ : not normal (so neither Hermitian nor unitary) — the standard nilpotent counterexample.

Exercise 13.3 ★

Prove that a matrix $A \in \mathcal{M}_n(\C)$ writes uniquely $A = H + \iu K$ with $H, K$ Hermitian (the “real and imaginary parts” $H = \frac{A + A^\dagger}{2}$ , $K = \frac{A - A^\dagger}{2\iu}$ ), and that $A$ is normal iff $H$ and $K$ commute.

Solution

Solution of Exercise 13.3.

Uniqueness: $A = H + \iu K$ with $H^\dagger = H$ , $K^\dagger = K$ forces $A^\dagger = H - \iu K$ , so $H = \frac{A + A^\dagger}{2}$ , $K = \frac{A - A^\dagger}{2\iu}$ ; these formulas are Hermitian (check: $\bigl(\frac{A - A^\dagger}{2\iu}\bigr)^\dagger = \frac{ A^\dagger - A}{-2\iu} = K$ ) and reconstruct $A$ : existence.

Normality: $A^\dagger A - AA^\dagger = (H - \iu K)(H + \iu K) - (H + \iu K)(H - \iu K) = 2\iu(HK - KH)$ : it vanishes iff $HK = KH$ .

Exercise 13.4 ★★

Diagonalize in an orthonormal basis: $A = \begin{pmatrix} 2 & \iu\\ -\iu & 2\end{pmatrix}$ , and compute $A^k$ for $k \in \N$ .

Solution

Solution of Exercise 13.4.

$\chi_A = (X-2)^2 - 1$ : eigenvalues $3$ and $1$ . Eigenvectors, by direct computation:

A\begin{pmatrix}1\\ -\iu\end{pmatrix} = \begin{pmatrix} 2 + \iu(-\iu)\\ -\iu + 2(-\iu)\end{pmatrix} = \begin{pmatrix} 3\\ -3\iu \end{pmatrix} = 3\begin{pmatrix}1\\ -\iu\end{pmatrix}, \qquad A\begin{pmatrix}1\\ \iu\end{pmatrix} = \begin{pmatrix} 2 + \iu\cdot\iu\\ -\iu + 2\iu\end{pmatrix} = \begin{pmatrix}1\\ \iu\end{pmatrix}.

Orthonormal eigenbasis: $u_1 = \frac{1}{\sqrt2}(1, -\iu)$ (eigenvalue $3$ ), $u_2 = \frac{1}{\sqrt2}(1, \iu)$ (eigenvalue $1$ ); orthogonality as in Exercise 13.1. Powers, via the spectral projections $A^k = 3^k u_1u_1^\dagger + 1^k\, u_2u_2^\dagger$ :

A^k = U\begin{pmatrix} 3^k & 0\\ 0 & 1\end{pmatrix}U^\dagger = \frac{3^k}{2}\begin{pmatrix} 1 & \iu\\ -\iu & 1\end{pmatrix} + \frac{1}{2}\begin{pmatrix} 1 & -\iu\\ \iu & 1\end{pmatrix} = \frac12\begin{pmatrix} 3^k + 1 & (3^k - 1)\iu\\ -(3^k-1)\iu & 3^k + 1 \end{pmatrix}.

(Check $k = 1$ : recovers $A$ .)

Exercise 13.5 ★★

Prove that $U(n)$ is compact, and that the eigenvalue map is onto: every unimodular $\lambda$ arises for some unitary matrix. Prove that $\det U \in \mathbb{U}$ (the unit circle) for $U \in U(n)$ .

Solution

Solution of Exercise 13.5.

Compact: closed (preimage of $I$ under the continuous $U \mapsto U^\dagger U$ ) and bounded (columns are unit vectors: entries of modulus $\leq 1$ ) in $\mathcal{M}_n(\C) \simeq \R^{2n^2}$ .

Eigenvalues: $\operatorname{diag}(\lambda, 1, \dots, 1)$ is unitary for any $\abs\lambda = 1$ . Determinant: $\abs{\det U}^2 = \det U^\dagger \det U = \det(U^\dagger U) = 1$ (using $\det A^\dagger = \conj{\det A}$ ): $\det U$ lies on the unit circle.

Exercise 13.6 ★★

(Cayley transform) Let $H$ be Hermitian. Prove that $H + \iu I$ is invertible and that $U = (H - \iu I)(H + \iu I)^{-1}$ is unitary, with $1 \notin \operatorname{Sp}(U)$ . (Work spectrally: on an eigenbasis of $H$ , everything is scalar.)

Solution

Solution of Exercise 13.6.

By the spectral theorem, work in an orthonormal eigenbasis of $H$ : everything reduces to scalars $\lambda \in \R$ (the eigenvalues). $H + \iu I$ has eigenvalues $\lambda + \iu \neq 0$ : invertible. $U$ has eigenvalues $\mu = \frac{\lambda - \iu}{\lambda + \iu}$ , of modulus $1$ ( $\abs{\lambda - \iu} = \abs{\lambda + \iu}$ for real $\lambda$ ): $U^\dagger U = I$ holds since $U$ is unitarily diagonalizable with unimodular eigenvalues (it is diagonal in the chosen orthonormal basis). And $\mu = 1$ would force $-\iu = \iu$ : impossible, so $1 \notin \operatorname{Sp} U$ . (The Cayley transform maps Hermitian to unitary-minus-a-point — the matrix version of the map from $\R$ to the circle.)

Exercise 13.7 ★★

For $A$ Hermitian positive definite ( $\langle x, Ax\rangle > 0$ for $x \neq 0$ ), prove that $\operatorname{Sp}(A) \subseteq \intoo{0}{\infty}$ , that $A = B^2$ for a Hermitian positive definite $B$ , and that $\det A > 0$ .

Solution

Solution of Exercise 13.7.

For an eigenpair $Ax = \lambda x$ ( $x \neq 0$ ): $\lambda\norm x^2 = \langle x, Ax\rangle > 0$ , so $\lambda > 0$ (already real, Proposition 13.6). Square root: in a spectral basis, $B = U\operatorname{diag}(\sqrt{\lambda_i})U^\dagger$ : Hermitian, positive definite, $B^2 = A$ . Determinant: product of the positive eigenvalues.

Exercise 13.8 ★★★

(Spectral theorem for normal endomorphisms) Let $u$ be normal on a Hermitian space.

Prove $\norm{u(x)} = \norm{u^*(x)}$ for all $x$ , and deduce $\ker(u - \lambda) = \ker(u^* - \conj\lambda)$ .
Prove that eigenspaces of $u$ for distinct eigenvalues are orthogonal, and that the orthogonal complement of an eigenspace is $u$ -stable.
Conclude by induction that $u$ is unitarily diagonalizable; and conversely.

Solution

Solution of Exercise 13.8.

$\norm{u(x)}^2 = \langle u(x), u(x)\rangle = \langle x, u^*u(x)\rangle = \langle x, uu^*(x)\rangle = \norm{u^*(x)}^2$ . Applying this to the normal $u - \lambda\,\mathrm{id}$ (its adjoint is $u^* - \conj\lambda$ , and normality is inherited): $\norm{(u - \lambda)x} = \norm{(u^* - \conj\lambda)x}$ , so the kernels agree.
For eigenvectors $u(x) = \lambda x$ , $u(y) = \mu y$ ( $\lambda \neq \mu$ ): using (1), $u^*(x) = \conj\lambda x$ ; then

$\lambda\langle y, x\rangle = \langle y, u(x)\rangle = \langle u^*(y), x\rangle = \langle \conj\mu\, y, x\rangle = \mu \langle y, x\rangle ,$

so $\langle y, x\rangle = 0$ . Stability of $E_\lambda^\perp$ : for $x \perp E_\lambda$ and $z \in E_\lambda$ , $\langle z, u(x)\rangle = \langle u^*(z), x\rangle = \langle\conj\lambda z, x\rangle = 0$ .
Induction on dimension: over $\C$ , $u$ has an eigenvector $e_1$ (normalize); its orthogonal complement is stable under $u$ (by (2)) and under $u^*$ (same argument with roles swapped), so the restriction is normal: induct and concatenate orthonormal eigenbases. Conversely, a unitarily diagonalizable $u = UDU^\dagger$ satisfies $u^*u = U\conj D D U^\dagger = U D\conj D U^\dagger = uu^*$ : normal.

Exercise 13.9 ★

An endomorphism is skew-Hermitian when $u^* = -u$ . Prove that its eigenvalues are purely imaginary, that $u \mapsto \iu u$ is a bijection from Hermitian to skew-Hermitian endomorphisms, and that skew-Hermitian endomorphisms are unitarily diagonalizable (Exercise 13.8).

Solution

Solution of Exercise 13.9.

Eigenvalues: for $u(x) = \lambda x$ , $x \neq 0$ :

\lambda\norm x^2 = \langle x, u(x)\rangle = \langle u^*(x), x\rangle = -\langle u(x), x\rangle = -\conj{\langle x, u(x)\rangle} = -\conj\lambda\,\norm x^2 ,

so $\lambda = -\conj\lambda$ : purely imaginary. Since $(\iu u)^* = -\iu\,u^*$ (the adjoint is conjugate-linear in scalars), $u^* = u$ gives $(\iu u)^* = -\iu u$ : the map $u \mapsto \iu u$ sends Hermitian to skew-Hermitian, with inverse $w \mapsto -\iu w$ : a bijection. A skew-Hermitian $u$ satisfies $u^*u = -u^2 = uu^*$ : normal, hence unitarily diagonalizable by Exercise 13.8.

Exercise 13.10 ★★

(The finite Fourier transform) Let $S$ be the cyclic shift of $\C^n$ : $S(x_0, x_1, \dots, x_{n-1}) = (x_{n-1}, x_0, \dots, x_{n-2})$ , and $\omega = \eu^{2\iu\pi/n}$ .

Show that $S$ is unitary, and that the vectors $f_k = \frac{1}{\sqrt n}\bigl(1, \omega^k, \omega^{2k}, \dots, \omega^{(n-1)k}\bigr)$ , $0 \leq k < n$ , form an orthonormal basis of eigenvectors: $Sf_k = \omega^{-k} f_k$ .
Deduce that every circulant matrix $C = \sum_{j=0}^{n-1} c_jS^j$ is normal, diagonalized by the same basis, with eigenvalues $\widehat c(k) = \sum_j c_j\,\omega^{-jk}$ .

Solution

Solution of Exercise 13.10.

$S$ permutes an orthonormal basis: $\norm{Sx} = \norm x$ , so $S$ is unitary. Indexing coordinates by $j = 0, \dots, n-1$ modulo $n$ : $(Sx)_j = x_{j-1}$ , so for $(f_k)_j = \frac{\omega^{jk}}{\sqrt n}$ :

$(Sf_k)_j = \frac{\omega^{(j-1)k}}{\sqrt n} = \omega^{-k}\,(f_k)_j : \qquad Sf_k = \omega^{-k}f_k .$

Orthonormality: $\langle f_k, f_l\rangle = \frac1n \sum_j \omega^{j(l-k)} = \delta_{kl}$ (geometric sum of a nontrivial root of unity vanishes).
$Cf_k = \sum_j c_j S^jf_k = \bigl(\sum_j c_j\omega^{-jk}\bigr)f_k = \widehat c(k)\,f_k$ : every circulant is diagonal in the orthonormal Fourier basis, hence normal, with spectrum $\{\widehat c(k)\}$ . (The change of basis is the discrete Fourier transform: convolution becomes multiplication.)

Exercise 13.11 ★★

Let $P$ be an idempotent ( $P^2 = P$ ) endomorphism of a Hermitian space. Prove that $P$ is the orthogonal projection onto $\operatorname{im} P$ if and only if $P^* = P$ . Give the matrix of the orthogonal projection onto $\C v$ ( $\norm v = 1$ ), and onto a subspace with orthonormal basis $(v_1, \dots, v_k)$ .

Solution

Solution of Exercise 13.11.

( $\Leftarrow$ ) Let $P^2 = P = P^*$ . Every $v$ splits as $v = Pv + (v - Pv)$ with $Pv \in \operatorname{im} P$ and $P(v - Pv) = 0$ . The two pieces are orthogonal: for any $x, y$ ,

\langle Px, (I - P)y\rangle = \langle x, P(I-P)y\rangle = \langle x, (P - P^2)y\rangle = 0 :

$\ker P \perp \operatorname{im} P$ , so $P$ is the orthogonal projection onto its image. ( $\Rightarrow$ ) If $P$ is the orthogonal projection onto $F = \operatorname{im}P$ : for all $x, y$ , $\langle Px, y\rangle = \langle Px, Py\rangle$ (the component $y - Py \perp F$ drops) and symmetrically $\langle x, Py\rangle = \langle Px, Py\rangle$ : $\langle Px, y\rangle = \langle x, Py\rangle$ , i.e. $P^* = P$ . Matrices: onto $\C v$ ( $\norm v = 1$ ): $Px = v\,\langle v, x\rangle$ , i.e. $P = vv^\dagger$ ; onto $\operatorname{Vect}(v_1, \dots, v_k)$ orthonormal: $P = \sum_i v_iv_i^\dagger$ .

Exercise 13.12 ★★★

(Spectral projectors by interpolation) Let $A$ be Hermitian with distinct eigenvalues $\lambda_1, \dots, \lambda_p$ and eigenspace decomposition $E = \bigoplus_i E_i$ . Define the Lagrange polynomials $L_i(X) = \prod_{j\neq i}\frac{X - \lambda_j}{\lambda_i - \lambda_j}$ . Prove that $P_i = L_i(A)$ is the orthogonal projection onto $E_i$ , that $P_iP_j = 0$ for $i \neq j$ , $\sum_i P_i = I$ , and $A = \sum_i \lambda_iP_i$ (the spectral decomposition); express $f(A)$ for any polynomial $f$ in terms of the $P_i$ .

Solution

Solution of Exercise 13.12.

Diagonalize $A = U D U^\dagger$ (spectral theorem), $D$ diagonal with entries among the $\lambda_i$ . Then $P_i = L_i(A) = U L_i(D)U^\dagger$ , and $L_i(D)$ is diagonal with entries $L_i(\lambda_j) = \delta_{ij}$ : ones exactly at the slots of $E_i$ . So $P_i$ is Hermitian ( $L_i$ real, $D$ real), idempotent, with image $E_i$ and kernel $\bigoplus_{j\neq i}E_j = E_i^\perp$ (orthogonality of eigenspaces): the orthogonal projection onto $E_i$ (Exercise 13.11). Disjoint diagonal patterns give $P_iP_j = 0$ ( $i \neq j$ ); $\sum_i L_i = 1$ (degree $< p$ , value $1$ at $p$ points), so $\sum P_i = I$ ; and $\sum_i \lambda_iL_i(\lambda_j) = \lambda_j$ gives $A = \sum \lambda_iP_i$ . For any polynomial $f$ : $f(D)$ has diagonal $f(\lambda_j)$ , so

f(A) = \sum_{i=1}^{p} f(\lambda_i)\,P_i :

functions of $A$ are computed spectrally — the calculus that the Year 3 volume extends to continuous $f$ and beyond.

13.4 Problem: Courant–Fischer, Weyl, and the calculus of eigenvalues

Problem 13.1

The eigenvalues of a Hermitian matrix are not just roots of a polynomial: they are solutions of optimization problems. That variational point of view — Rayleigh quotients and the Courant–Fischer min-max theorem — makes eigenvalues comparable, stable, and computable, and this problem harvests its classical crops: Weyl’s perturbation inequalities, Cauchy interlacing, the Schur and Ky Fan trace inequalities, the monotonicity of the matrix square root, and the spectrum of the discrete Laplacian. Throughout, $A, B, E$ are Hermitian on $E = \C^n$ with eigenvalues listed in decreasing order $\lambda_1(A) \geq \dots \geq \lambda_n(A)$ , and $R_A(x) = \frac{\langle x, Ax\rangle}{\langle x, x\rangle}$ for $x \neq 0$ is the Rayleigh quotient.

Part I — Rayleigh quotients and min-max. Fix an orthonormal eigenbasis $(e_1, \dots, e_n)$ , $Ae_i = \lambda_ie_i$ .

Show that $R_A(x)$ is real, and that

$\lambda_n \leq R_A(x) \leq \lambda_1 \qquad (x \neq 0),$

both bounds attained: $\lambda_1 = \max R_A$ , $\lambda_n = \min R_A$ .
Show that the critical points of $R_A$ are exactly the eigenvectors of $A$ (expand $t \mapsto R_A(x + tv)$ at $t = 0$ for $v$ arbitrary, then replace $v$ by $\iu v$ ).
Let $V_k = \operatorname{Vect}(e_1, \dots, e_k)$ and $W_k = \operatorname{Vect}(e_k, \dots, e_n)$ . Show

$\min_{x \in V_k\setminus\{0\}} R_A(x) = \lambda_k = \max_{x \in W_k\setminus\{0\}} R_A(x) .$
Prove the Courant–Fischer theorem: for $1 \leq k \leq n$ ,

$\lambda_k = \max_{\dim V = k}\;\min_{x \in V\setminus\{0\}} R_A(x) = \min_{\dim W = n-k+1}\;\max_{x \in W\setminus\{0\}} R_A(x)$

(for any $V$ of dimension $k$ : $V \cap W_k \neq \{0\}$ by Grassmann, so $\min_V R_A \leq \lambda_k$ ; question 3 shows the bound is attained).
(Monotonicity) Write $A \leq B$ when $B - A$ is positive semidefinite. Deduce from question 4: $A \leq B$ implies $\lambda_k(A) \leq \lambda_k(B)$ for every $k$ .

Part II — Weyl’s inequalities.

Show that subspaces $V, W \subseteq \C^n$ with $\dim V + \dim W > n$ intersect nontrivially, and generalize: $\dim(V_1 \cap V_2 \cap V_3) \geq \dim V_1 + \dim V_2 + \dim V_3 - 2n$ .
Prove Weyl’s inequality: for $i + j - 1 \leq n$ ,

$\lambda_{i+j-1}(A + B) \leq \lambda_i(A) + \lambda_j(B)$

(intersect the subspaces $W_i(A)$ , $W_j(B)$ and $V_{i+j-1}(A+B)$ of question 3 and count dimensions).
Define $\vertiii{E}_2 = \max_{\norm x = 1}\norm{Ex}$ and show $\vertiii E_2 = \max_k\abs{\lambda_k(E)}$ for Hermitian $E$ . Deduce Weyl’s perturbation theorem:

$\bigl|\lambda_k(A + E) - \lambda_k(A)\bigr| \leq \vertiii{E}_2 \qquad (1 \leq k \leq n) :$

each eigenvalue is a $1$ -Lipschitz function of the matrix.
(Rank-one perturbations) Let $P$ be Hermitian positive semidefinite of rank $1$ . Show

$\lambda_k(A) \leq \lambda_k(A + P) \leq \lambda_{k-1}(A) \qquad (2 \leq k \leq n),$

together with $\lambda_1(A) \leq \lambda_1(A+P)$ : the new eigenvalues interlace the old ones.
Check question 8 numerically: $A = \begin{pmatrix} 2 & \iu\\ -\iu & 2\end{pmatrix}$ (Exercise 13.4: spectrum $\{3, 1\}$ ) and $E = \begin{pmatrix} 0 & 1\\ 1 & 0\end{pmatrix}$ (spectrum $\{1, -1\}$ ): compute the spectrum of $A + E$ and both sides of the inequality.

Part III — Interlacing and trace inequalities.

(Cauchy interlacing) Let $B$ be the leading $(n-1) \times(n-1)$ principal submatrix of $A$ . Prove

$\lambda_{k+1}(A) \leq \lambda_k(B) \leq \lambda_k(A) \qquad (1 \leq k \leq n-1)$

(view $\C^{n-1} \subseteq \C^n$ ; on it, $R_B$ is the restriction of $R_A$ ; apply Courant–Fischer on both levels).
Iterate: for a principal submatrix $B$ of size $n - m$ , $\lambda_{k+m}(A) \leq \lambda_k(B) \leq \lambda_k(A)$ .
(Schur) Let $d_1 \geq d_2 \geq \dots \geq d_n$ be the diagonal entries of $A$ , sorted. Prove, for every $k$ :

$\sum_{i=1}^{k} d_i \leq \sum_{i=1}^{k}\lambda_i(A),$

with equality at $k = n$ (the trace) (the $k$ chosen diagonal entries form a principal $k\times k$ submatrix; bound its trace by question 12).
(Ky Fan) Prove

$\sum_{i=1}^{k}\lambda_i(A) = \max\Bigl\{\sum_{i=1}^{k}\langle x_i, Ax_i\rangle : (x_1, \dots, x_k) \text{ orthonormal}\Bigr\} .$
Verify questions 11 and 13 on

$A = \begin{pmatrix} 2 & 1 & 0\\ 1 & 2 & 1\\ 0 & 1 & 2\end{pmatrix} \qquad \bigl(\operatorname{Sp} = \{2 - \sqrt2,\ 2,\ 2+\sqrt2\}\bigr)$

against its leading $2\times2$ block (spectrum $\{1, 3\}$ ) and its diagonal.

Part IV — The Loewner order. $A \leq B$ still means $B - A$ positive semidefinite; all matrices in this part are Hermitian.

Show: $A \leq B$ implies $a_{ii} \leq b_{ii}$ for all $i$ , $\operatorname{tr} A \leq \operatorname{tr} B$ , and $C^\dagger AC \leq C^\dagger BC$ for every complex matrix $C$ .
Show that squaring is not monotone: for

$A = \begin{pmatrix} 1 & 0\\ 0 & 0\end{pmatrix}, \qquad B = \begin{pmatrix} 2 & 1\\ 1 & 1\end{pmatrix},$

check $0 \leq A \leq B$ but $A^2 \not\leq B^2$ .
Prove that the square root is monotone: $0 \leq A \leq B$ implies $\sqrt A \leq \sqrt B$ (let $\mu$ be an eigenvalue of $\sqrt B - \sqrt A$ with unit eigenvector $v$ ; compute $\langle v, (B - A)v\rangle = \mu\bigl(\langle v, \sqrt B\,v\rangle + \langle v, \sqrt A\,v\rangle\bigr)$ and discuss).
Prove that inversion is antitone on positive definite matrices: $0 < A \leq B$ implies $B^{-1} \leq A^{-1}$ (congruate by $A^{-1/2}$ to reduce to $I \leq M \Rightarrow M^{-1} \leq I$ , which is scalar in a spectral basis).
Let $A, B$ be positive definite. Show that the eigenvalues of $AB$ (not Hermitian in general!) are real and positive, and that

$\lambda_{\max}(AB) \leq \lambda_{\max}(A)\,\lambda_{\max}(B)$

(conjugate by $\sqrt A$ : $AB \sim \sqrt A\,B\sqrt A$ ).

Part V — The discrete Laplacian, worked. Let $T_n$ be the $n \times n$ tridiagonal matrix with $2$ on the diagonal and $-1$ on the two adjacent diagonals.

With $\theta_k = \frac{k\pi}{n+1}$ , verify that the vectors $v_k = \bigl(\sin(j\theta_k)\bigr)_{1\leq j\leq n}$ satisfy $T_nv_k = (2 - 2\cos\theta_k)\,v_k$ (product-to-sum identity; check the boundary rows $j = 1, n$ ). Conclude:

$\operatorname{Sp}(T_n) = \Bigl\{4\sin^2 \frac{k\pi}{2(n+1)} : 1 \leq k \leq n\Bigr\},$

all simple, all positive: $T_n$ is positive definite.
(A potential) For a real diagonal $D = \operatorname{diag}(d_1, \dots, d_n)$ , sandwich the spectrum: for every $k$ ,

$\lambda_k(T_n) + \min_i d_i \;\leq\; \lambda_k(T_n + D) \;\leq\; \lambda_k(T_n) + \max_i d_i .$
Check Cauchy interlacing between $T_3$ and $T_2$ explicitly (spectra $\{2 \pm \sqrt2, 2\}$ and $\{1, 3\}$ ), and interpret: $T_2$ is $T_3$ with one endpoint of the path removed.
Show the extreme eigenvalues satisfy, as $n \to \infty$ :

$\lambda_{\min}(T_n) = 4\sin^2\frac{\pi}{2(n+1)} \sim \frac{\pi^2}{(n+1)^2}, \qquad \lambda_{\max}(T_n) \to 4 ,$

so the condition number $\kappa_n = \lambda_{\max}/\lambda_{\min}$ grows like $\frac{4(n+1)^2}{\pi^2}$ : discretizing a second derivative on a finer and finer grid is intrinsically ill-conditioned.
Synthesis. In one sentence each: (i) why the variational characterization, not the characteristic polynomial, is what makes eigenvalues stable (questions 8–9); (ii) which questions used only $\lambda_1 = \max R_A$ and which needed the full min-max; (iii) what the Loewner order adds to the story; (iv) where these tools reappear (numerical analysis of question 24’s stiffness matrices; quantum perturbation theory; and, in the Year 3 volume, the min-max principle for compact self-adjoint operators).

Solution

Solution of Problem 13.1.

1. $\conj{\langle x, Ax\rangle} = \langle Ax, x\rangle = \langle x, A^*x\rangle = \langle x, Ax\rangle$ : real. Writing $x = \sum c_ie_i$ :

R_A(x) = \frac{\sum_i\lambda_i\abs{c_i}^2} {\sum_i\abs{c_i}^2} ,

a weighted average of the eigenvalues: it lies in $\intcc{\lambda_n}{\lambda_1}$ , with the bounds attained at $e_1$ and $e_n$ .

2. For real $t$ and any $v$ , expand $R_A(x + tv) = \frac{N(t)}{D(t)}$ with

N(t) = \langle x, Ax\rangle + 2t\Re\langle v, Ax\rangle + t^2\langle v, Av\rangle, \quad D(t) = \norm x^2 + 2t\Re\langle v, x\rangle + t^2\norm v^2 .

The derivative at $t = 0$ is

\frac{2}{\norm x^2}\, \Re\bigl\langle v,\ Ax - R_A(x)\,x\bigr\rangle .

It vanishes for all $v$ iff $\Re\langle v, w\rangle = 0$ for all $v$ , where $w = Ax - R_A(x)x$ ; replacing $v$ by $\iu v$ kills the imaginary part too: $w = 0$ , i.e. $Ax = R_A(x)x$ . Critical points of $R_A$ are exactly eigenvectors, with critical value the eigenvalue.

3. For $x = \sum_{i\leq k}c_ie_i \in V_k$ : $R_A(x)$ is a weighted average of $\lambda_1, \dots, \lambda_k$ , hence $\geq \lambda_k$ , with equality at $e_k$ : $\min_{V_k} R_A = \lambda_k$ . Symmetrically on $W_k$ the average involves $\lambda_k, \dots, \lambda_n$ : $\max_{W_k}R_A = \lambda_k$ .

4. Let $\dim V = k$ . Then $\dim V + \dim W_k = n + 1 > n$ , so there is a unit $x \in V \cap W_k$ , and $R_A(x) \leq \lambda_k$ (question 3): $\min_{V}R_A \leq \lambda_k$ for every such $V$ . Since $V_k$ attains $\lambda_k$ , the max-min equals $\lambda_k$ . The min-max formula is the same argument with the roles reversed ( $\dim W = n - k + 1$ forces $W \cap V_k \neq \{0\}$ , so $\max_W R_A \geq \lambda_k$ , attained at $W_k$ ).

5. $R_B(x) = R_A(x) + \frac{\langle x, (B-A)x\rangle}{\norm x^2} \geq R_A(x)$ pointwise. Taking $\min$ over any $k$ -dimensional $V$ and then $\max$ over $V$ : $\lambda_k(B) \geq \lambda_k(A)$ by question 4.

6. Grassmann: $\dim(V\cap W) = \dim V + \dim W - \dim(V + W) \geq \dim V + \dim W - n > 0$ . Applying this twice:

\dim(V_1\cap V_2\cap V_3) \geq \dim(V_1\cap V_2) + \dim V_3 - n \geq \dim V_1 + \dim V_2 + \dim V_3 - 2n .

7. The subspaces $W_i(A)$ , $W_j(B)$ (question 3, for $A$ and $B$ ) and $V_{i+j-1}(A+B)$ have dimensions $(n-i+1) + (n-j+1) + (i+j-1) = 2n + 1 > 2n$ : by question 6 there is a unit vector $x$ in all three. Then

\lambda_{i+j-1}(A+B) \leq R_{A+B}(x) = R_A(x) + R_B(x) \leq \lambda_i(A) + \lambda_j(B),

the left inequality because $x \in V_{i+j-1}(A+B)$ (question 3), the right by the two $W$ ’s.

8. In a spectral basis of $E$ : $\norm{Ex}^2 = \sum \lambda_k(E)^2\abs{c_k}^2 \leq \max_k\lambda_k(E)^2\,\norm x^2$ , attained at the corresponding eigenvector: $\vertiii E_2 = \max_k\abs{\lambda_k(E)}$ . Weyl with $j = 1$ : $\lambda_k(A+E) \leq \lambda_k(A) + \lambda_1(E) \leq \lambda_k(A) + \vertiii E_2$ ; applying this to $(A+E) + (-E)$ : $\lambda_k(A) \leq \lambda_k(A+E) + \vertiii E_2$ . Together: $\abs{\lambda_k(A+E) - \lambda_k(A)} \leq \vertiii E_2$ .

9. Lower bounds: $P \geq 0$ and question 5. Upper: $P$ has rank $1$ , so $\lambda_2(P) = 0$ ; Weyl with $i = k-1$ , $j = 2$ :

\lambda_k(A + P) \leq \lambda_{k-1}(A) + \lambda_2(P) = \lambda_{k-1}(A) .

10. $A + E = \begin{pmatrix} 2 & 1+\iu\\ 1-\iu & 2\end{pmatrix}$ : characteristic polynomial $(2-\lambda)^2 - \abs{1+\iu}^2 = (2-\lambda)^2 - 2$ , spectrum $\{2 + \sqrt2,\ 2 - \sqrt2\}$ . Against $\operatorname{Sp}A = \{3, 1\}$ :

\abs{(2+\sqrt2) - 3} = \abs{(2-\sqrt2) - 1} = \sqrt2 - 1 \approx 0.414 \leq 1 = \vertiii E_2 . \checkmark

11. View $\C^{n-1} = \operatorname{Vect}(e_1, \dots, e_{n-1})$ inside $\C^n$ (standard basis): for $x$ there, $\langle x, Bx\rangle = \langle x, Ax\rangle$ , so $R_B$ is the restriction of $R_A$ . Upper bound: the max-min for $\lambda_k (B)$ ranges over $k$ -dimensional subspaces of $\C^{n-1}$ , a subfamily of those of $\C^n$ : $\lambda_k(B) \leq \lambda_k(A)$ . Lower bound: the min-max for $\lambda_k(B)$ ranges over subspaces of $\C^{n-1}$ of dimension $(n-1)-k+1 = n-k$ ; each is also a subspace of $\C^n$ of dimension $n - (k+1) + 1$ , so its max is $\geq \lambda_{k+1}(A)$ : $\lambda_k(B) \geq \lambda_{k+1}(A)$ .

12. Remove the rows/columns one at a time and chain question 11: each removal shifts the lower index by one, giving $\lambda_{k+m}(A) \leq \lambda_k(B) \leq \lambda_k(A)$ .

13. Conjugating $A$ by a permutation matrix (unitary) neither changes the spectrum nor the multiset of diagonal entries: assume $d_1, \dots, d_k$ occupy the leading positions. The leading $k\times k$ principal submatrix $B$ then has $\operatorname{tr} B = \sum_{i\leq k}d_i$ , and its eigenvalues satisfy $\mu_i(B) \leq \lambda_i(A)$ (question 12): summing, $\sum_{i\leq k}d_i \leq \sum_{i\leq k}\lambda_i(A)$ . At $k = n$ both sides are $\operatorname{tr} A$ .

14. Taking $x_i = e_i$ gives the value $\sum_{i\leq k}\lambda_i$ : the max is $\geq$ . Conversely, an orthonormal family $(x_1, \dots, x_k)$ extends to an orthonormal basis, i.e. to a unitary $U$ with first columns $x_i$ ; then $\sum_i\langle x_i, Ax_i\rangle$ is the sum of the first $k$ diagonal entries of $U^\dagger AU$ , which by question 13 is at most the sum of its $k$ largest eigenvalues — namely $\sum_{i\leq k}\lambda_i(A)$ . Ky Fan’s maximum principle follows.

15. Interlacing ( $\operatorname{Sp}A = \{2+\sqrt2, 2, 2-\sqrt2\}$ , $\operatorname{Sp}B = \{3, 1\}$ ):

2 \leq 3 \leq 2 + \sqrt2, \qquad 2 - \sqrt2 \leq 1 \leq 2 . \checkmark

Schur with diagonal $(2,2,2)$ : $2 \leq 2+\sqrt2$ ; $4 \leq 4 + \sqrt2$ ; $6 = 6$ (trace). ✓16. $b_{ii} - a_{ii} = \langle e_i, (B-A)e_i\rangle \geq 0$ ; summing gives the traces. For any $C$ : $\langle x, C^\dagger(B - A)Cx\rangle = \langle Cx, (B-A)(Cx)\rangle \geq 0$ : $C^\dagger AC \leq C^\dagger BC$ .

17. $A \geq 0$ clear; $B - A = \begin{pmatrix} 1 & 1\\ 1 & 1\end{pmatrix}$ is positive semidefinite (eigenvalues $2, 0$ ): $A \leq B$ . But

B^2 = \begin{pmatrix} 5 & 3\\ 3 & 2\end{pmatrix}, \qquad B^2 - A^2 = \begin{pmatrix} 4 & 3\\ 3 & 2\end{pmatrix}, \qquad \det(B^2 - A^2) = -1 < 0 :

not positive semidefinite. Squaring does not respect the Loewner order.

18. Let $S = \sqrt A$ , $T = \sqrt B$ (Hermitian positive semidefinite, Exercise 13.7 extended to semidefinite by the same spectral formula). $T - S$ is Hermitian; let $\mu$ be any eigenvalue, $v$ a unit eigenvector. From $T^2 - S^2 = T(T - S) + (T - S)S$ :

0 \leq \langle v, (B - A)v\rangle = \langle Tv, (T-S)v\rangle + \langle (T-S)v, Sv\rangle = \mu\bigl(\langle v, Tv\rangle + \langle v, Sv\rangle\bigr)

( $\mu$ is real). If $\langle v, Tv\rangle + \langle v, Sv\rangle > 0$ , then $\mu \geq 0$ . If it vanishes, both nonnegative terms vanish; $\langle v, Tv\rangle = \norm{T^{1/2}v}^2 = 0$ forces $Tv = 0$ , likewise $Sv = 0$ , so $\mu v = (T - S)v = 0$ and $\mu = 0$ . All eigenvalues of $T - S$ are $\geq 0$ : $\sqrt A \leq \sqrt B$ .

19. Congruence by $A^{-1/2}$ (question 16): $I \leq M := A^{-1/2}BA^{-1/2}$ . So all eigenvalues of $M$ are $\geq 1$ , hence those of $M^{-1}$ lie in $\intoc{0}{1}$ : $M^{-1} \leq I$ . But $M^{-1} = A^{1/2}B^{-1}A^{1/2}$ ; congruating $M^{-1} \leq I$ by $A^{-1/2}$ gives $B^{-1} \leq A^{-1}$ .

20. $\sqrt A^{-1}(AB)\sqrt A = \sqrt A\,B\sqrt A$ : so $AB$ is similar to the Hermitian positive definite $\sqrt A\,B\sqrt A$ (definite: $\langle x, \sqrt AB\sqrt Ax\rangle = \langle \sqrt Ax, B\sqrt Ax\rangle > 0$ ): its eigenvalues are real and positive. Moreover

\langle x, \sqrt AB\sqrt Ax\rangle \leq \lambda_{\max}(B)\,\norm{\sqrt Ax}^2 = \lambda_{\max}(B)\,\langle x, Ax\rangle \leq \lambda_{\max}(A)\lambda_{\max}(B)\norm x^2 ,

so $\lambda_{\max}(AB) = \max R_{\sqrt AB\sqrt A} \leq \lambda_{\max}(A)\lambda_{\max}(B)$ .

21. With $v_k = (\sin j\theta_k)_j$ and the identity $\sin((j-1)\theta) + \sin((j+1)\theta) = 2\sin(j\theta)\cos\theta$ : for $2 \leq j \leq n-1$ ,

(T_nv_k)_j = -\sin((j{-}1)\theta_k) + 2\sin(j\theta_k) - \sin((j{+}1)\theta_k) = (2 - 2\cos\theta_k)\sin(j\theta_k) .

Row $1$ works because $\sin(0\cdot\theta_k) = 0$ , row $n$ because $\sin((n+1)\theta_k) = \sin(k\pi) = 0$ : the boundary conditions select exactly $\theta_k = \frac{k\pi}{n+1}$ . So $T_nv_k = 4\sin^2\bigl(\frac{k\pi}{2(n+1)}\bigr)v_k$ ; the $n$ values are distinct in $\intoo04$ and the $v_k \neq 0$ : this is the whole spectrum, positive, so $T_n$ is positive definite.

22. $\min_id_i\,I \leq D \leq \max_id_i\,I$ , so $T_n + \min_id_i\,I \leq T_n + D \leq T_n + \max_id_i\,I$ (adding $T_n$ preserves the order); question 5 and $\lambda_k(T_n + cI) = \lambda_k(T_n) + c$ give the sandwich.

23. $T_2 = \begin{pmatrix} 2 & -1\\ -1 & 2\end{pmatrix}$ has spectrum $\{3, 1\}$ , and

2 - \sqrt2 \;\leq\; 1 \;\leq\; 2 \;\leq\; 3 \;\leq\; 2 + \sqrt2 :

Cauchy interlacing, verified. (These are the same spectra as in question 15: conjugating by $\operatorname{diag}(1,-1,1)$ flips the sign of the off-diagonal.) Graph reading: $T_2$ is the Laplacian-type matrix of the path with the last vertex deleted — a principal submatrix, exactly the situation of question 11.

24. The extreme eigenvalues behave as

\lambda_{\min}(T_n) = 4\sin^2\frac{\pi}{2(n+1)} \sim \frac{\pi^2}{(n+1)^2}, \qquad \lambda_{\max}(T_n) = 4\cos^2\frac{\pi}{2(n+1)} \longrightarrow 4 .

Hence

\kappa_n = \frac{\lambda_{\max}}{\lambda_{\min}} \sim \frac{4(n+1)^2}{\pi^2} :

the finer the grid, the worse conditioned the discrete second derivative — a fact that drives the design of numerical linear algebra.

25. (i) Roots of the characteristic polynomial can move wildly under perturbations of a general matrix, but the min-max characterization pins each Hermitian eigenvalue between explicit optimization values, forcing the $1$ -Lipschitz stability of questions 8–9. (ii) Questions 1, 5, 16–20 used only the extreme Rayleigh values; Weyl, interlacing, Schur and Ky Fan (questions 7–14) genuinely needed the full min-max over subspaces. (iii) The Loewner order turns these scalar inequalities into a calculus of matrix inequalities — with real traps (question 17) and real theorems (questions 18–19). (iv) These tools are the daily bread of numerical analysis (question 24’s stiffness matrices), of quantum perturbation theory (Weyl: energy levels move by at most the norm of the perturbation), and of the Year 3 volume’s min-max principle for compact self-adjoint operators.