University Mathematics — Year 2 · Bachelor Year 2

15Differential Calculus

The two-variable calculus of the Year 1 volume matures into the differential calculus of maps between normed spaces: the differential as the best linear approximation, the chain rule in full generality, Schwarz’s symmetry theorem proved, Taylor formulas, and the complete second-order analysis of extrema. The inverse function theorem, crown of the theory, is stated with its proof strategy — a Banach fixed point — made explicit.

Throughout, $U$ is an open subset of $\R^n$ (or a normed space; the finite-dimensional case carries all the ideas), $f \colon U \to \R^m$ .

15.1 The differential

Definition 15.1

$f$ is differentiable at $a$ when there is a (continuous) linear map $\dd f_a \colon \R^n \to \R^m$ with

f(a + h) = f(a) + \dd f_a(h) + o(\norm h) \qquad (h \to 0).

The map $\dd f_a$ , the differential of $f$ at $a$ , is unique; its matrix in the canonical bases is the Jacobian matrix $J_f(a) = \bigl(\frac{\partial f_i}{\partial x_j}(a)\bigr)$ . Differentiability implies continuity and the existence of all directional derivatives $\dd f_a(v) = \lim_{t\to0}\frac{f(a + tv) - f(a)}{t}$ ; the converse fails (Exercise 15.2). For $m = 1$ , $\dd f_a(h) = \langle \nabla f(a), h\rangle$ : the gradient of Year 1, now understood as the vector representing the differential.

Theorem 15.2 ( $C^1$ criterion)

If all partial derivatives of $f$ exist on $U$ and are continuous at $a$ , then $f$ is differentiable at $a$ . “ $C^1$ on $U$ ” — continuous partials — therefore implies differentiability everywhere, with continuous differential.

Proof. Component by component ( $m = 1$ suffices). The Year 1 proof for two variables — move one coordinate at a time, apply the one-variable mean value theorem on each leg, and use continuity of the partials at $a$ — generalizes verbatim to $n$ legs:

f(a + h) - f(a) = \sum_{j=1}^{n} \bigl(f(a + h^{(j)}) - f(a + h^{(j-1)})\bigr) = \sum_j h_j\,\frac{\partial f}{\partial x_j}(\xi_j),

where $h^{(j)}$ freezes the first $j$ coordinates of $h$ and $\xi_j$ lies on the $j$ -th leg; continuity turns each $\frac{\partial f}{\partial x_j}(\xi_j)$ into $\frac{\partial f}{\partial x_j}(a) + o(1)$ , and the error is $o(\norm h)$ . ∎

Theorem 15.3 (Chain rule)

If $f$ is differentiable at $a$ and $g$ at $f(a)$ , then $g \circ f$ is differentiable at $a$ with

\dd(g \circ f)_a = \dd g_{f(a)} \circ \dd f_a , \qquad J_{g\circ f}(a) = J_g\bigl(f(a)\bigr)\,J_f(a) :

Jacobians multiply.

Proof. Write $f(a + h) = f(a) + \dd f_a(h) + \norm h\,\varepsilon_1(h)$ and $g(b + k) = g(b) + \dd g_b(k) + \norm k\,\varepsilon_2(k)$ with $b = f(a)$ , $k = k(h) = \dd f_a(h) + \norm h \varepsilon_1(h)$ . Substituting,

g(f(a+h)) = g(b) + \dd g_b\bigl(\dd f_a(h)\bigr) + \norm h\,\dd g_b(\varepsilon_1(h)) + \norm{k}\,\varepsilon_2(k),

and both error terms are $o(\norm h)$ : the first since $\dd g_b$ is continuous and $\varepsilon_1 \to 0$ ; the second since $\norm k \leq C\norm h$ (bounded linear map plus small term) and $\varepsilon_2(k) \to 0$ as $h \to 0$ . ∎

Example 15.4 (Radial functions, once and for all)

Let $r(x) = \norm x_2$ on $\R^n\setminus\{0\}$ and $f = g \circ r$ with $g$ a $C^1$ function of one variable. First, $r$ is differentiable away from $0$ : from $r^2 = \sum x_i^2$ ,

\frac{\partial r}{\partial x_i} = \frac{x_i}{r}, \qquad\text{i.e.}\qquad \nabla r(x) = \frac{x}{\norm x} ,

the unit radial vector (differentiate $r^2$ and divide — or apply the chain rule to $\sqrt{\cdot}$ ). Then the chain rule gives, for every radial function,

\nabla f(x) = g'\bigl(\norm x\bigr)\,\frac{x}{\norm x} .

Worked instance: $g(r) = \frac1r$ yields $\nabla\frac{1}{\norm x} = -\frac{x}{\norm x^3}$ , the inverse-square field of gravitation and electrostatics — direction radial, magnitude $\frac{1}{\norm x^2}$ . Closing insight: gradients of radial functions are radial because the level sets are spheres and the gradient is orthogonal to level sets; at $x = 0$ , by contrast, $r$ is not differentiable (no candidate linear map matches $\norm h$ from all directions) — smooth radial profiles need $g'(0) = 0$ to cross the origin gracefully.

Theorem 15.5 (Mean value inequality)

Let $f$ be differentiable on $U$ and let the segment $\intcc{a}{b} = \{a + t(b-a)\}$ lie in $U$ . Then

\norm{f(b) - f(a)} \leq \norm{b - a}\, \sup_{x \in \intcc{a}{b}} \vertiii{\dd f_x} .

In particular, a differentiable map with zero differential on a connected open set is constant.

Proof. The function $\varphi(t) = f(a + t(b-a))$ is differentiable on $\intcc{0}{1}$ with $\varphi'(t) = \dd f_{a + t(b-a)}(b - a)$ (chain rule), of norm $\leq M\norm{b-a}$ with $M$ the displayed sup. For $\R$ -valued $f$ the one-variable mean value inequality concludes; for vector values apply it to $t \mapsto \langle u, \varphi(t)\rangle$ with $u$ the unit vector along $f(b) - f(a)$ . Constancy: locally constant (segments in balls) plus connectedness (the set where $f$ equals a given value is open and closed: Chapter 4). ∎

Example 15.6 (A Lipschitz constant from the MVI)

Is $f(x, y) = \sin x\,\sin y$ Lipschitz on $\R^2$ , and with which constant? Its gradient is $\nabla f = (\cos x\sin y,\ \sin x\cos y)$ , of squared norm

\cos^2x\sin^2y + \sin^2x\cos^2y \leq \sin^2 y + \cos^2y\cdot 1 = 1

(bound $\cos^2x$ and $\sin^2x$ by $1$ separately), so $\vertiii{\dd f_{(x,y)}} = \norm{\nabla f} \leq 1$ everywhere, and Theorem 15.5 on the segment between any two points gives

\abs{f(b) - f(a)} \leq \norm{b - a}_2 :

$f$ is $1$ -Lipschitz, and the constant is sharp (near the origin, $f(x, \tfrac\pi2) = \sin x$ has slope $1$ ). Closing insight: the mean value inequality converts a pointwise bound on the differential into a global modulus of continuity — the standard route to Lipschitz estimates in every dimension, and the engine inside Exercise 15.12.

15.2 Second derivatives

Theorem 15.7 (Schwarz)

If $f$ is $C^2$ on $U$ (all second partials exist and are continuous), then for all $i, j$ :

\frac{\partial^2 f}{\partial x_i\,\partial x_j} = \frac{\partial^2 f}{\partial x_j\,\partial x_i} .

Proof. Two variables suffice ( $x = x_i$ , $y = x_j$ , others frozen). Consider the second difference

\Delta(h) = f(a + h, b + h) - f(a + h, b) - f(a, b + h) + f(a,b) .

Fix $h$ and set $\varphi(x) = f(x, b+h) - f(x, b)$ : then $\Delta(h) = \varphi(a + h) - \varphi(a)$ , and two applications of the mean value theorem give

\Delta(h) = h\,\varphi'(\xi) = h\Bigl(\frac{\partial f}{\partial x}(\xi, b+h) - \frac{\partial f}{\partial x}(\xi, b)\Bigr) = h^2\,\frac{\partial^2 f}{\partial y\,\partial x}(\xi, \eta),

with $\xi \in \intoo{a}{a+h}$ , $\eta \in \intoo{b}{b+h}$ . By continuity, $\frac{\Delta(h)}{h^2} \to \frac{\partial^2 f}{\partial y\partial x}(a, b)$ as $h \to 0$ . The same computation with the roles of the variables exchanged (freeze the second variable first) gives $\frac{\Delta(h)}{h^2} \to \frac{\partial^2 f}{\partial x \partial y}(a,b)$ : the two limits of the same quantity coincide. ∎

Example 15.8 (Why $C^2$ is needed: Peano’s counterexample)

Let $f(x, y) = \dfrac{xy(x^2 - y^2)}{x^2 + y^2}$ , $f(0,0) = 0$ . Away from the origin $f$ is $C^\infty$ ; at the origin all first and second partials exist, but the mixed ones disagree. Compute along the axes: $f(x, 0) = f(0, y) = 0$ , and for $y \neq 0$ ,

\frac{\partial f}{\partial x}(0, y) = \lim_{x\to0}\frac{f(x,y)}{x} = \frac{y(0 - y^2)}{y^2} = -y , \qquad\text{symmetrically}\qquad \frac{\partial f}{\partial y}(x, 0) = x .

Hence

\frac{\partial^2 f}{\partial y\,\partial x}(0,0) = \frac{\dd}{\dd y}\Bigl[\frac{\partial f}{\partial x}(0,y)\Bigr]_{y=0} = -1, \qquad \frac{\partial^2 f}{\partial x\,\partial y}(0,0) = +1 :

the two mixed partials exist and differ. No contradiction with Theorem 15.7: the second partials of $f$ are not continuous at $0$ (test along $y = tx$ ). Closing insight: Schwarz’s theorem is a genuine theorem about continuity, not a formal identity — and the hypothesis “ $C^2$ ” in Taylor–Young below is doing real work.

Theorem 15.9 (Taylor–Young at order 2)

Let $f \colon U \to \R$ be $C^2$ and $a \in U$ . Then, as $h \to 0$ ,

f(a + h) = f(a) + \langle\nabla f(a), h\rangle + \frac12\, \langle H_a h,\, h\rangle + o\bigl(\norm h^2\bigr),

where $H_a = \bigl(\frac{\partial^2 f}{\partial x_i\partial x_j}(a)\bigr)$ is the (symmetric, by Schwarz) Hessian matrix.

Proof. Apply the one-variable Taylor–Young theorem (Year 1 volume) to $\varphi(t) = f(a + th)$ on $\intcc{0}{1}$ : by the chain rule, $\varphi'(t) = \langle \nabla f(a + th), h\rangle$ and $\varphi''(t) = \langle H_{a+th}h, h\rangle$ , both continuous in $t$ . Then $\varphi(1) = \varphi(0) + \varphi'(0) + \frac12\varphi''(\theta)$ (Taylor–Lagrange) with $\theta \in \intoo{0}{1}$ , and continuity of the second partials converts $\varphi''(\theta) = \varphi''(0) + o(1)\cdot\norm h^2$ -uniformly: the displayed expansion. ∎

Example 15.10 (An expansion two ways)

Expand $f(x, y) = \eu^x\cos y$ at the origin to order $2$ . By composition of one-variable expansions:

\eu^x\cos y = \Bigl(1 + x + \frac{x^2}{2} + o(x^2)\Bigr)\Bigl(1 - \frac{y^2}{2} + o(y^2)\Bigr) = 1 + x + \frac{x^2 - y^2}{2} + o\bigl(\norm{(x,y)}^2\bigr) .

By partial derivatives: $f_x = \eu^x\cos y$ , $f_y = -\eu^x\sin y$ , so $\nabla f(0) = (1, 0)$ ; and $f_{xx} = f$ , $f_{yy} = -f$ , $f_{xy} = -\eu^x\sin y$ give $H_0 = \operatorname{diag}(1, -1)$ : Theorem 15.9 reproduces $1 + x + \frac12(x^2 - y^2)$ . The two computations agree, and the composition route was faster — no second partials at all. Closing insight: the origin is not a critical point ( $\nabla f \neq 0$ ), so despite the indefinite Hessian there is no saddle to declare: the linear term rules, and the second-order test only ever speaks at critical points.

Theorem 15.11 (Second-order extremum test, proved)

Let $f$ be $C^2$ near a critical point $a$ ( $\nabla f(a) = 0$ ), with Hessian $H = H_a$ .

If $H$ is positive definite, $a$ is a strict local minimum (negative definite: maximum).
If $H$ has eigenvalues of both signs, $a$ is a saddle: no extremum.
If $H$ is singular (and semidefinite), no conclusion.

The Year 1 “ $rt - s^2$ ” test is the case $n = 2$ : $\det H = rt - s^2$ , $\operatorname{tr}$ -sign read off $r$ .

Proof. By the spectral theorem (Theorem 12.13), $H$ ’s quadratic form is squeezed between its extreme eigenvalues: $\lambda_{\min}\norm h^2 \leq \langle Hh, h\rangle \leq \lambda_{\max}\norm h^2$ .

(1) If $\lambda_{\min} > 0$ : Taylor–Young gives

f(a + h) - f(a) \geq \frac{\lambda_{\min}}{2}\norm h^2 - o(\norm h^2) > 0

for small $h \neq 0$ : strict local minimum.

(2) Along an eigenvector $v_+$ with $\lambda_+ > 0$ : $f(a + tv_+) - f(a) = \frac{\lambda_+}{2}t^2 + o(t^2) > 0$ for small $t$ ; along $v_-$ with $\lambda_- < 0$ the difference is negative: both signs occur in every neighborhood.

(3) $f(x,y) = x^2 + y^4$ , $x^2 - y^4$ , $x^2 + y^3$ share the same singular semidefinite Hessian at $0$ with three different behaviors. ∎

Example 15.12 (A complete classification, global included)

Classify all extrema of $f(x, y) = x^4 + y^4 - 4xy$ on $\R^2$ . Critical points: $\nabla f = (4x^3 - 4y,\ 4y^3 - 4x) = 0$ gives $y = x^3$ and $x = y^3 = x^9$ , so $x(x^8 - 1) = 0$ : the real solutions are $(0,0)$ , $(1,1)$ , $(-1,-1)$ . Hessians: $H = \begin{pmatrix} 12x^2 & -4\\ -4 & 12y^2\end{pmatrix}$ . At $(\pm1, \pm1)$ : $\begin{pmatrix} 12 & -4\\ -4 & 12\end{pmatrix}$ , eigenvalues $8$ and $16$ : positive definite, strict local minima with $f = -2$ . At $(0,0)$ : $\begin{pmatrix} 0 & -4\\ -4 & 0\end{pmatrix}$ , eigenvalues $\pm4$ : a saddle. Globality: from $2\abs{xy} \leq x^2 + y^2$ ,

f(x, y) \geq x^4 + y^4 - 2(x^2 + y^2) = (x^2 - 1)^2 + (y^2 - 1)^2 + x^2 + y^2 - 2 \xrightarrow[\norm{(x,y)}\to\infty]{} +\infty :

$f$ is coercive, so it attains a global minimum (compactness of sublevel sets), necessarily at a critical point: the value $-2$ , at both $(1,1)$ and $(-1,-1)$ , is the global minimum; there is no maximum ( $f$ unbounded above). Closing insight: the local test classifies candidates, but only a growth argument turns “local” into “global” — the two-step pattern of every optimization proof in this book.

Method 15.13 (Classifying the extrema of $f \colon \R^n \to \R$ )

Solve $\nabla f = 0$ (all critical points; on a domain with boundary, treat the boundary separately as in Exercise 15.7).
At each critical point, compute the Hessian and its eigenvalue signs — in dimension $2$ , just $\det H$ and $\operatorname{tr} H$ : $\det < 0$ saddle; $\det > 0$ extremum, of the type given by the sign of the trace; $\det = 0$ : the test is silent, study $f$ along curves.
For global statements, add a compactness or coercivity argument ( $f \to +\infty$ at infinity, or a compact constraint set), then compare the critical values.

15.3 The inverse function theorem

Theorem 15.14 (Inverse function theorem)

Let $f \colon U \to \R^n$ be $C^1$ and $a \in U$ with $\dd f_a$ invertible. Then there are open neighborhoods $V \ni a$ , $W \ni f(a)$ such that $f \colon V \to W$ is a bijection with $C^1$ inverse, and

\dd (f^{-1})_{f(x)} = (\dd f_x)^{-1} \qquad (x \in V).

Proof. Admitted at this level. ∎

Remark 15.15 (Why it is true: the fixed-point strategy)

Solving $f(x) = y$ near $a$ rewrites as the fixed-point equation $x = x + \dd f_a^{-1}\bigl(y - f(x)\bigr) =: \Phi_y(x)$ ; the map $\Phi_y$ has differential $\mathrm{id} - \dd f_a^{-1}\dd f_x$ , small near $a$ by continuity of $\dd f$ , so $\Phi_y$ is a contraction on a small closed ball and the Banach fixed point theorem (Theorem 4.12) provides the unique local solution $x = f^{-1}(y)$ . Continuity and differentiability of the inverse then follow from the estimates in the contraction. The full bookkeeping is carried out in Year 3; the strategy — and the statement — are used freely from now on. The companion implicit function theorem (solving $F(x, y) = 0$ for $y(x)$ when $\frac{\partial F}{\partial y}$ is invertible) follows by applying the theorem to $(x, y) \mapsto (x, F(x,y))$ .

Example 15.16 (Polar coordinates)

$\Phi(r, \theta) = (r\cos\theta, r\sin\theta)$ has Jacobian

J_\Phi = \begin{pmatrix} \cos\theta & -r\sin\theta\\ \sin\theta & r\cos\theta \end{pmatrix}, \qquad \det J_\Phi = r :

invertible for $r \neq 0$ , so $\Phi$ is a local $C^1$ diffeomorphism away from the origin — the licence to “change to polar coordinates”, renewed for the multiple integrals of Chapter 20.

Example 15.17 (Local everywhere, global nowhere)

Let $f(x, y) = \bigl(\eu^x\cos y,\ \eu^x\sin y\bigr)$ on $\R^2$ . Its Jacobian,

J_f = \begin{pmatrix} \eu^x\cos y & -\eu^x\sin y\\ \eu^x\sin y & \eu^x\cos y \end{pmatrix}, \qquad \det J_f = \eu^{2x} > 0 ,

never vanishes: by Theorem 15.14, $f$ is a local $C^1$ diffeomorphism at every point of the plane. Yet $f$ is far from injective: $f(x, y + 2\pi) = f(x, y)$ , so every value is taken infinitely often; and it is not surjective either, since $\norm{f(x, y)} = \eu^x > 0$ misses the origin. Closing insight: the inverse function theorem is irreducibly local — invertibility of every $\dd f_a$ yields a patchwork of local inverses that need not assemble into one. (Readers who know complex numbers will recognize $z \mapsto \eu^z$ ; the patchwork is the family of logarithm branches.) Compare Exercise 15.12, where a quantitative global hypothesis does force one global inverse.

Remark 15.18 (Common pitfalls)

(i) Directional derivatives are cheap, differentials are not: all directional derivatives may exist — and even fail to depend linearly on the direction — without differentiability (Exercise 15.2); only the $C^1$ criterion (Theorem 15.2) upgrades partials to a differential. (ii) Critical does not mean extremal: saddles (Example 15.12) and the silent singular case (Theorem 15.11 (3)) both hide behind $\nabla f = 0$ . (iii) No vector-valued mean value equality: only the inequality of Theorem 15.5 survives (Exercise 15.9); never write $f(b) - f(a) = \dd f_c(b-a)$ for $f$ with values in $\R^m$ , $m \geq 2$ . (iv) Local invertibility is not injectivity: Example 15.17. (v) The gradient belongs to the inner product: $\nabla f$ is the vector representing $\dd f_a$ in a chosen inner product; change the product (as in the weighted example of the quadratic-forms chapter) and the gradient rotates, while the differential — the intrinsic object — does not move.

Remark 15.19 (Where this is used)

Everything downstream of this chapter is differential calculus applied: the differential-equations chapter linearizes flows and uses Liouville’s determinant formula (proved in this chapter’s weekend problem); the chapters on curves and surfaces study level sets and parametrizations through the implicit function theorem; multiple integrals change variables through Jacobians. The weekend problem develops the calculus on the space of matrices itself — differential of the determinant, of the inverse, the matrix exponential, and the orthogonal group as a smooth level set — the Year 2 shadow of what the Year 3 volume formalizes as manifolds and Lie groups.

15.4 Exercises

Exercise 15.1 ★

Compute the Jacobian matrices of $f(x,y) = (x^2 - y^2,\, 2xy)$ and of $\Phi(r,\theta,z) = (r\cos\theta, r\sin\theta, z)$ ; where are the differentials invertible?

Solution

Solution of Exercise 15.1.

$J_f = \begin{pmatrix} 2x & -2y\\ 2y & 2x\end{pmatrix}$ , $\det J_f = 4(x^2 + y^2)$ : invertible away from the origin. (This $f$ is $z \mapsto z^2$ in complex disguise.)

$J_\Phi = \begin{pmatrix} \cos\theta & -r\sin\theta & 0\\ \sin\theta & r\cos\theta & 0\\ 0 & 0 & 1\end{pmatrix}$ , $\det = r$ : invertible for $r \neq 0$ (cylindrical coordinates).

Exercise 15.2 ★

Let $f(x,y) = \frac{x^3}{x^2 + y^2}$ ( $f(0,0) = 0$ ). Prove that all directional derivatives of $f$ at $0$ exist, but that $f$ is not differentiable at $0$ (the map $v \mapsto$ directional derivative is not linear).

Solution

Solution of Exercise 15.2.

For $v = (a, b) \neq 0$ : $\frac{f(tv) - 0}{t} = \frac{t^3a^3}{t\cdot t^2(a^2+b^2)} = \frac{a^3}{a^2 + b^2}$ : every directional derivative exists, with value $D_v = \frac{a^3}{a^2+b^2}$ . But $v \mapsto D_v$ is not linear ( $D_{(1,0)} = 1$ , $D_{(0,1)} = 0$ , $D_{(1,1)} = \frac12 \neq 1$ ): no linear map can produce these values, so $f$ is not differentiable at $0$ (the differential would have to be $v \mapsto D_v$ ).

Exercise 15.3 ★

Find and classify the critical points of $f(x, y) = x^3 + y^3 - 3xy$ using Theorem 15.11, and of $g(x,y) = x^4 + y^4 - 2(x - y)^2$ .

Solution

Solution of Exercise 15.3.

$f = x^3 + y^3 - 3xy$ : critical points $(0,0)$ and $(1,1)$ (Year 1 computation). Hessians: $H = \begin{pmatrix} 6x & -3\\ -3 & 6y\end{pmatrix}$ . At $(0,0)$ : eigen-signs mixed ( $\det = -9 < 0$ ): saddle. At $(1,1)$ : $\det = 27 > 0$ , trace $> 0$ : positive definite, strict local minimum — now justified by Theorem 15.11 rather than decreed.

$g = x^4 + y^4 - 2(x-y)^2$ : $\nabla g = (4x^3 - 4(x - y),\; 4y^3 + 4(x-y))$ ; critical points $(0,0)$ , $(\sqrt2, -\sqrt2)$ , $(-\sqrt2, \sqrt2)$ (Year 1). At $(\pm\sqrt2, \mp\sqrt2)$ : $H = \begin{pmatrix} 12\cdot2 - 4 & 4\\ 4 & 20\end{pmatrix} = \begin{pmatrix} 20 & 4\\ 4 & 20\end{pmatrix}$ : positive definite (diagonally dominant; eigenvalues $24, 16$ ): strict local minima. At $(0,0)$ : $H = \begin{pmatrix} -4 & 4\\ 4 & -4\end{pmatrix}$ , singular negative semidefinite: the test is silent; the directional study ( $g(x,x) = 2x^4 > 0$ , $g(x,-x) = 2x^4 - 8x^2 < 0$ small) shows a saddle-like point — no extremum.

Exercise 15.4 ★★

Let $f \colon \R^n \to \R$ be $C^1$ and homogeneous of degree $p$ : $f(tx) = t^pf(x)$ for $t > 0$ . Prove Euler’s identity

\langle \nabla f(x), x\rangle = p\,f(x) ,

and its converse for $C^1$ functions on $\R^n\setminus\{0\}$ .

Solution

Solution of Exercise 15.4.

Differentiate $t \mapsto f(tx)$ at $t = 1$ : by the chain rule, $\langle \nabla f(x), x\rangle$ ; by homogeneity the same function is $t^pf(x)$ , of derivative $pf(x)$ at $t = 1$ : Euler’s identity.

Converse: fix $x \neq 0$ and let $\varphi(t) = f(tx) - t^p f(x)$ on $t > 0$ . Then $\varphi'(t) = \langle\nabla f(tx), x\rangle - pt^{p-1}f(x) = \frac1t\bigl(\langle \nabla f(tx), tx\rangle - p\,t^pf(x)\bigr)$ . The hypothesis — Euler’s identity at the point $tx$ — evaluates the bracket as $p\,f(tx) - p\,t^pf(x) = p\,\varphi(t)$ . So $\varphi' = \frac{p}{t}\varphi$ with $\varphi(1) = 0$ : the linear ODE’s unique solution is $\varphi \equiv 0$ (Year 1 uniqueness), i.e. $f(tx) = t^pf(x)$ .

Exercise 15.5 ★★

Let $A$ be symmetric and $f(x) = \frac12\langle Ax, x\rangle - \langle b, x\rangle$ . Compute $\nabla f$ and $H_f$ ; when is $f$ convex? Assuming $A$ positive definite, show that $f$ has a unique global minimum at the solution of $Ax = b$ — gradient descent’s raison d’être.

Solution

Solution of Exercise 15.5.

Expanding $f(x + h) - f(x) = \langle Ax - b, h\rangle + \frac12\langle Ah, h\rangle$ (symmetry of $A$ ): $\nabla f(x) = Ax - b$ and $H_f = A$ everywhere. $f$ is convex iff $A$ is positive semidefinite (the Hessian test, global here since $H$ is constant: the second-order Taylor formula is exact). If $A$ is positive definite: the unique critical point is $x^* = A^{-1}b$ , and $f(x^* + h) - f(x^*) = \frac12\langle Ah, h\rangle \geq \frac{\lambda_{\min}}{2}\norm h^2 > 0$ for $h \neq 0$ : strict global minimum.

Exercise 15.6 ★★

(Lagrange multiplier, one constraint, proved by hand) Let $f, g$ be $C^1$ on $\R^2$ , and suppose $f$ attains, at $a$ , a local extremum on the level set $\{g = 0\}$ , with $\nabla g(a) \neq 0$ . Prove that $\nabla f(a) = \lambda\nabla g(a)$ for some $\lambda$ . (Parametrize the level set near $a$ by the implicit function theorem and differentiate $t \mapsto f(\gamma(t))$ .) Application: extrema of $f(x,y) = xy$ on the circle $x^2 + y^2 = 1$ .

Solution

Solution of Exercise 15.6.

Since $\nabla g(a) \neq 0$ , one partial, say $\frac{\partial g}{\partial y}(a) \neq 0$ : the implicit function theorem (Theorem 15.14’s companion) parametrizes $\{g = 0\}$ near $a = (a_1, a_2)$ as $\gamma(t) = (t, y(t))$ with $y$ $C^1$ , $y'(t) = -\frac{\partial_x g}{\partial_y g}(\gamma(t))$ (differentiate $g(t, y(t)) = 0$ ). The one-variable function $t \mapsto f(\gamma(t))$ has a local extremum at $t = a_1$ :

0 = \frac{\dd}{\dd t}f(\gamma(t))\Big|_{a_1} = \partial_x f(a) + \partial_y f(a)\,y'(a_1) = \partial_x f(a) - \partial_yf(a)\frac{\partial_x g(a)}{\partial_y g(a)} :

the vectors $\nabla f(a)$ and $\nabla g(a)$ have proportional coordinates: $\nabla f(a) = \lambda \nabla g(a)$ with $\lambda = \frac{\partial_y f(a)}{\partial_y g(a)}$ .

Application: on the circle, $\nabla(xy) = (y, x)$ parallel to $(2x, 2y)$ forces $y^2 = x^2$ ; with the constraint, the candidates are $\pm\bigl(\tfrac{1}{\sqrt2}, \tfrac{1}{\sqrt2}\bigr)$ (value $\frac12$ ) and $\pm\bigl(\tfrac{1}{\sqrt2}, -\tfrac{1}{\sqrt2}\bigr)$ (value $-\frac12$ ): max $\frac12$ , min $-\frac12$ (attained: the circle is compact).

Exercise 15.7 ★★

Determine the extrema of $f(x, y) = x^2 + y^2 - xy + x - y$ on $\R^2$ , then its maximum and minimum on the closed triangle with vertices $(0,0)$ , $(1,0)$ , $(0,1)$ (interior critical points, then the three edges, then the vertices).

Solution

Solution of Exercise 15.7.

$\nabla f = (2x - y + 1,\; 2y - x - 1) = 0$ : solving, $x = -\frac13$ , $y = \frac13$ . Hessian $\begin{pmatrix} 2 & -1\\ -1 & 2\end{pmatrix}$ , positive definite: global minimum of the quadratic $f$ , value $f\bigl(-\frac13, \frac13\bigr) = -\frac13$ .

On the triangle $T$ : the interior critical point $(-\frac13, \frac13) \notin T$ (negative $x$ ). Edges: on $y = 0$ , $x \in \intcc{0}{1}$ : $f = x^2 + x$ , increasing: extremes $0$ and $2$ . On $x = 0$ : $f = y^2 - y$ , minimum $-\frac14$ at $y = \frac12$ , values $0$ and $0$ at the ends. On $x + y = 1$ : substitute $y = 1 - x$ , $f = x^2 + (1-x)^2 - x(1-x) + x - (1-x) = 3x^2 - x$ ; on $\intcc{0}{1}$ : minimum $-\frac{1}{12}$ at $x = \frac16$ , values $0$ (at $x=0$ ) and $2$ (at $x=1$ ). Vertices: $f(0,0) = 0$ , $f(1,0) = 2$ , $f(0,1) = 0$ . Global on $T$ : minimum $-\frac14$ at $(0, \frac12)$ , maximum $2$ at $(1, 0)$ .

Exercise 15.8 ★★★

Let $f \colon \R^2 \to \R^2$ , $f(x, y) = (x + y^2,\; y + x^2)$ . Show that $f$ is a local diffeomorphism near $0$ , compute $\dd(f^{-1})_{(0,0)}$ , and find the largest $r$ such that $\dd f$ is invertible on the ball $\norm{(x,y)}_2 < r$ (compute $\det J_f$ ).

Solution

Solution of Exercise 15.8.

$J_f = \begin{pmatrix} 1 & 2y\\ 2x & 1\end{pmatrix}$ , $\det J_f = 1 - 4xy$ . At $0$ : $\det = 1 \neq 0$ : local diffeomorphism (Theorem 15.14), with

\dd(f^{-1})_{(0,0)} = (J_f(0))^{-1} = I_2 .

Invertibility on a ball: need $4\abs{xy} < 1$ throughout; on $\norm{(x,y)}_2 < r$ , $\abs{xy} \leq \frac{x^2 + y^2}{2} < \frac{r^2}{2}$ , so $r = \frac{1}{\sqrt2}$ works; and it is largest: at $(x, y) = \bigl(\tfrac12, \tfrac12\bigr)$ , of norm $\frac{1}{\sqrt2}$ , $\det J_f = 0$ .

Exercise 15.9 ★★★

(Rolle fails, mean value survives) Give $f \colon \R \to \R^2$ , $C^1$ , with $f(0) = f(2\pi)$ but $f'(t) \neq 0$ for all $t$ (no vector-valued Rolle). Then verify on your example the mean value inequality of Theorem 15.5.

Solution

Solution of Exercise 15.9.

$f(t) = (\cos t, \sin t)$ : $f(0) = f(2\pi) = (1, 0)$ , yet $f'(t) = (-\sin t, \cos t)$ has norm $1$ , never zero: no point where the derivative vanishes — Rolle has no vector analogue. The mean value inequality holds comfortably: $\norm{f(2\pi) - f(0)} = 0 \leq 2\pi \cdot \sup\norm{f'} = 2\pi$ .

Exercise 15.10 ★

Compute the differential and the gradient of $f(x) = \norm x_2^2$ and of $g(x) = \langle Ax, x\rangle$ on $\R^n$ ( $A$ a square matrix, not assumed symmetric), and the Hessian of each. For which $A$ is $g$ convex?

Solution

Solution of Exercise 15.10.

$f(x+h) - f(x) = 2\langle x, h\rangle + \norm h^2$ : $\dd f_x = 2\langle x, \cdot\rangle$ , $\nabla f(x) = 2x$ , Hessian $2I$ (constant). For $g$ :

g(x + h) - g(x) = \langle Ax, h\rangle + \langle Ah, x\rangle + \langle Ah, h\rangle = \bigl\langle (A + A^{\mathsf T})x,\ h\bigr\rangle + O(\norm h^2),

so $\nabla g(x) = (A + A^{\mathsf T})x$ and $H_g = A + A^{\mathsf T}$ , constant. By Exercise 15.11, $g$ is convex iff $A + A^{\mathsf T}$ is positive semidefinite — only the symmetric part of $A$ matters, as indeed $g(x) = \langle \frac{A + A^{\mathsf T}}2 x, x\rangle$ .

Exercise 15.11 ★★

Let $f \colon \R^n \to \R$ be $C^2$ . Prove that $f$ is convex if and only if its Hessian $H_x$ is positive semidefinite at every $x$ (reduce to one variable: $t \mapsto f(a + t(b-a))$ ; use Taylor–Lagrange in one direction, and for the converse evaluate $\varphi''$ ).

Solution

Solution of Exercise 15.11.

$f$ is convex iff its restriction to every segment is convex, i.e. iff every $\varphi(t) = f(a + tv)$ is convex. By the chain rule $\varphi''(t) = \langle H_{a+tv}\,v,\ v\rangle$ .

If all Hessians are positive semidefinite: $\varphi'' \geq 0$ , so each $\varphi$ is convex (Year 1 volume) and $f$ is convex. Conversely if $f$ is convex, each $\varphi$ is convex, so $\varphi''(0) \geq 0$ : $\langle H_a v, v\rangle \geq 0$ for every $a$ and every direction $v$ : all Hessians are positive semidefinite.

Exercise 15.12 ★★★

(A global inverse theorem) Let $g \colon \R^n \to \R^n$ be $C^1$ with $\vertiii{\dd g_x} \leq k < 1$ for all $x$ , and $f = \mathrm{id} + g$ .

Show $\norm{f(x) - f(y)} \geq (1 - k)\norm{x - y}$ : $f$ is injective, with continuous inverse on its image.
Show that for each $y \in \R^n$ the map $x \mapsto y - g(x)$ is a contraction of the complete space $\R^n$ , and conclude by the Banach fixed point theorem (Theorem 4.12) that $f$ is surjective.
Conclude that $f$ is a bijection of $\R^n$ with $(1-k)^{-1}$ -Lipschitz inverse — a global counterpart of Theorem 15.14 (which, by contrast, is purely local).

Solution

Solution of Exercise 15.12.

By the mean value inequality (Theorem 15.5) applied to $g$ : $\norm{g(x) - g(y)} \leq k\norm{x-y}$ , so

$\norm{f(x) - f(y)} \geq \norm{x - y} - \norm{g(x) - g(y)} \geq (1 - k)\norm{x - y} :$

$f$ is injective and $f^{-1}$ (defined on the image) is $\frac{1}{1-k}$ -Lipschitz.
Fix $y$ ; $T(x) = y - g(x)$ satisfies $\norm{T(x) - T(x')} = \norm{g(x') - g(x)} \leq k\norm{x - x'}$ : a contraction of the complete space $\R^n$ . Banach (Theorem 4.12) gives a fixed point $x^* = y - g(x^*)$ , i.e. $f(x^*) = y$ : $f$ is surjective.
$f$ is thus a bijection of $\R^n$ with $\frac{1}{1-k}$ -Lipschitz inverse: a global inverse theorem, where the smallness of $\dd g$ everywhere replaces the local invertibility hypothesis of Theorem 15.14.

15.5 Problem: The calculus of matrices — Jacobi, exponential, and the orthogonal group

Problem 15.1

The cleanest playground for differential calculus is the space $\mathcal M_n(\R) \simeq \R^{n^2}$ itself: its most natural maps — product, inverse, determinant, exponential — have differentials of striking elegance. This problem computes them all: the Neumann series, the differential of the inverse, Jacobi’s formula for the determinant with Liouville’s formula as dividend, the matrix exponential with $\det\eu^A = \eu^{\operatorname{tr}A}$ , and finally the orthogonal group $O_n$ as a smooth level set with the antisymmetric matrices as tangent space — differential geometry in embryo. Throughout, $\vertiii\cdot$ is the operator norm subordinate to $\norm\cdot_2$ , and $\langle X, Y\rangle = \operatorname{tr}(X^{\mathsf T}Y)$ the Frobenius inner product.

Part I — The Neumann series.

Prove submultiplicativity, $\vertiii{AB} \leq \vertiii A\,\vertiii B$ , and deduce that polynomial maps of $A$ (matrix products, determinant, trace) are continuous on $\mathcal M_n(\R)$ .
For $\vertiii X < 1$ , show that $\sum_{k\geq0}X^k$ converges absolutely in $\mathcal M_n(\R)$ (Theorem 5.21), that its sum is $(I - X)^{-1}$ , and that

$\vertiii{(I - X)^{-1}} \leq \frac{1}{1 - \vertiii X}, \qquad (I - X)^{-1} = I + X + O\bigl(\vertiii X^2\bigr) .$
Deduce that $GL_n(\R)$ is open: if $A$ is invertible and $\vertiii H < \frac{1}{\vertiii{A^{-1}}}$ , then $A + H$ is invertible. Deduce also that $GL_n(\R)$ is dense in $\mathcal M_n(\R)$ (perturb $A$ by $\varepsilon I$ : $\det(A + \varepsilon I)$ is a nonzero polynomial in $\varepsilon$ ).
Show that the inversion map $\Phi(A) = A^{-1}$ is continuous on $GL_n(\R)$ .

Part II — First differentials.

Show that the squaring map $A \mapsto A^2$ is differentiable with differential $H \mapsto AH + HA$ , and more generally that $A \mapsto A^k$ has differential $H \mapsto \sum_{i=0}^{k-1} A^iHA^{k-1-i}$ . Why can one not write $kA^{k-1}H$ in general?
Prove that $\Phi(A) = A^{-1}$ is differentiable on $GL_n(\R)$ with

$\dd\Phi_A(H) = -A^{-1}HA^{-1}$

(write $(A + H)^{-1} = (I + A^{-1}H)^{-1}A^{-1}$ and expand by question 2). Check the formula against the scalar case $n = 1$ .
For a $C^1$ curve $t \mapsto A(t) \in GL_n(\R)$ , deduce $\bigl(A(t)^{-1}\bigr)' = -A^{-1}A'A^{-1}$ , and expand $t \mapsto (I + tB)^{-1}$ to first order at $t = 0$ .
Compute the differential of $f(A) = \operatorname{tr}(A^k)$ and identify its gradient for the Frobenius inner product:

$\dd f_A(H) = k\operatorname{tr}\bigl(A^{k-1}H\bigr), \qquad \nabla f(A) = k\,\bigl(A^{k-1}\bigr)^{\mathsf T} .$
Same questions for $f(A) = \operatorname{tr} (A^{\mathsf T}A) = \norm A_F^2$ : differential, gradient, and the (constant) Hessian; conclude that $\norm\cdot_F^2$ is strictly convex.

Part III — Jacobi’s formula.

Prove

$\det(I + H) = 1 + \operatorname{tr}H + O\bigl(\vertiii H^2\bigr)$

(expand $\det(e_1 + h_1, \dots, e_n + h_n)$ by multilinearity in the columns: terms with at least two $h$ -columns are $O(\vertiii H^2)$ ): $\dd(\det)_I = \operatorname{tr}$ .
For $A$ invertible, deduce

$\dd(\det)_A(H) = \det(A)\, \operatorname{tr}\bigl(A^{-1}H\bigr) .$
Show that for every $A$ (invertible or not), $\frac{\partial\det}{\partial a_{ij}}(A) = C_{ij}$ , the $(i,j)$ cofactor (Laplace expansion along row $i$ ), so that with the adjugate $\operatorname{adj}A = \operatorname{com}(A)^{\mathsf T}$ :

$\dd(\det)_A(H) = \operatorname{tr}\bigl(\operatorname{adj}(A)\,H\bigr), \qquad \nabla(\det)(A) = \operatorname{com}(A) ,$

recovering question 11 when $A$ is invertible ( $\operatorname{adj}A = \det(A)A^{-1}$ ). This is Jacobi’s formula: $\bigl(\det A(t)\bigr)' = \operatorname{tr}\bigl( \operatorname{adj}(A(t))\,A'(t)\bigr)$ .
(Liouville’s formula) Let $A(t)$ be a $C^1$ curve of matrices satisfying the linear differential equation $A'(t) = M(t)A(t)$ . Prove

$\bigl(\det A(t)\bigr)' = \operatorname{tr}\bigl(M(t)\bigr)\,\det A(t), \qquad\text{hence}\qquad \det A(t) = \det A(0)\, \exp\Bigl(\int_0^t\operatorname{tr}M\Bigr)$

(use $\operatorname{adj}(A)\,A = \det(A)I$ and the cyclic invariance of the trace) — the Wronskian identity that the differential-equations chapter will use constantly.
Show that $SL_n(\R) = \{\det = 1\}$ is a smooth level set: at every $A \in SL_n(\R)$ the differential $\dd(\det)_A$ is a surjective linear map onto $\R$ (evaluate it at $H = \frac1nA$ ).

Part IV — The matrix exponential.

Show that $\eu^A = \sum_{k\geq0}\frac{A^k}{k!}$ converges absolutely for every $A$ , normally on every ball, with $\vertiii{\eu^A} \leq \eu^{\vertiii A}$ ; and that $\eu^A$ depends continuously on $A$ .
Prove that $AB = BA$ implies $\eu^{A+B} = \eu^A\eu^B$ (Cauchy product, legitimate by absolute convergence); deduce that $\eu^A$ is always invertible, with inverse $\eu^{-A}$ : $\exp$ maps $\mathcal M_n(\R)$ into $GL_n(\R)$ .
Show that $t \mapsto \eu^{tA}$ is $C^1$ (indeed $C^\infty$ ) with

$\frac{\dd}{\dd t}\,\eu^{tA} = A\,\eu^{tA} = \eu^{tA}A$

(differentiate the series term by term on segments: the derived series converges normally).
Prove the identity

$\det\bigl(\eu^{A}\bigr) = \eu^{\operatorname{tr}A}$

(apply Liouville’s formula, question 13, to $A(t) = \eu^{tA}$ ). Sanity checks: $n = 1$ ; nilpotent $A$ ; and the trace-zero matrices land in $SL_n(\R)$ .
Show $\eu^H = I + H + O(\vertiii H^2)$ , so $\exp$ is differentiable at $0$ with $\dd(\exp)_0 = \mathrm{id}$ ; conclude with the inverse function theorem (Theorem 15.14) that $\exp$ is a $C^1$ diffeomorphism from a neighborhood of $0$ onto a neighborhood of $I$ : every matrix close to the identity has a logarithm.
Show that $\exp$ maps symmetric matrices to symmetric positive definite matrices, bijectively (diagonalize; the inverse is the spectral logarithm).

Part V — The orthogonal group as a level set.

Let $F(A) = A^{\mathsf T}A$ , from $\mathcal M_n(\R)$ to the symmetric matrices $S_n$ . Compute $\dd F_A(H) = A^{\mathsf T}H + H^{\mathsf T}A$ and show that at every $A \in O_n = F^{-1}(I)$ this differential is surjective onto $S_n$ (given $S \in S_n$ , try $H = \frac12 AS$ ): $O_n$ is a smooth level set, of dimension $n^2 - \frac{n(n+1)}2 = \frac{n(n-1)}2$ .
Show that every $C^1$ curve $A(t) \in O_n$ with $A(0) = I$ has an antisymmetric velocity $A'(0)$ , and conversely that for antisymmetric $K$ the curve $\eu^{tK}$ stays in $O_n$ : the tangent space of $O_n$ at $I$ is exactly the antisymmetric matrices.
Show $\det\eu^{K} = 1$ for antisymmetric $K$ (question 18): the exponential curve lives in the rotation group $SO_n$ . Compute it fully for $n = 2$ : with $J = \begin{pmatrix}0 & -1\\ 1 & 0\end{pmatrix}$ , prove

$\eu^{\theta J} = \begin{pmatrix} \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta\end{pmatrix} :$

the matrix exponential is the rotation by $\theta$ , and the series definitions of cosine and sine reappear inside a matrix.
(The density trick) Using the density of $GL_n(\R)$ (question 3) and continuity, extend from invertible to all matrices the identity

$\operatorname{adj}(AB) = \operatorname{adj}(B)\operatorname{adj}(A)$

(for invertible $A, B$ both sides equal $\det(AB)(AB)^{-1}$ ; both sides are polynomial in the entries).
Synthesis. In one sentence each: (i) which earlier chapters provided the engine of each Part (completeness and normed algebras; the spectral theorem; the inverse function theorem); (ii) which formula of this problem the differential-equations chapter will lean on, and where; (iii) what $\dd(\det)_I = \operatorname{tr}$ and $\det\eu^A = \eu^{\operatorname{tr}A}$ say about trace and determinant as “infinitesimal and global volume”; (iv) what the Year 3 volume makes of questions 21–23 (Lie groups and their Lie algebras).

Solution

Solution of Problem 15.1.

1. $\norm{ABx} \leq \vertiii A\norm{Bx} \leq \vertiii A\vertiii B\norm x$ : take the sup over $\norm x = 1$ . Matrix products, $\det$ and $\operatorname{tr}$ are polynomial functions of the entries, hence continuous ( $\mathcal M_n(\R) \simeq \R^{n^2}$ , all norms equivalent: Theorem 5.13).

2. $\sum\vertiii{X^k} \leq \sum\vertiii X^k < \infty$ : the series converges absolutely, hence converges (Theorem 5.21). From $(I - X)\sum_{k\leq N}X^k = I - X^{N+1} \to I$ : the sum is $(I - X)^{-1}$ . Norm: $\leq \sum\vertiii X^k = \frac{1}{1 - \vertiii X}$ ; and

(I - X)^{-1} - I - X = \sum_{k\geq2}X^k = X^2(I - X)^{-1}, \qquad \vertiii{X^2(I-X)^{-1}} \leq \frac{\vertiii X^2}{1 - \vertiii X} = O(\vertiii X^2).

3. $A + H = A(I + A^{-1}H)$ with $\vertiii{A^{-1}H} \leq \vertiii{A^{-1}}\vertiii H < 1$ : invertible by question 2: the open ball of radius $\vertiii{A^{-1}}^{-1}$ around $A$ lies in $GL_n(\R)$ . Density: $\det(A + \varepsilon I)$ is a degree- $n$ polynomial in $\varepsilon$ with leading coefficient $1$ : it has finitely many roots, so there are $\varepsilon_k \to 0$ with $A + \varepsilon_kI$ invertible, converging to $A$ .

4. For $\vertiii H < \frac{1}{2\vertiii{A^{-1}}}$ :

(A + H)^{-1} - A^{-1} = \bigl[(I + A^{-1}H)^{-1} - I\bigr]A^{-1},

of norm at most $\frac{\vertiii{A^{-1}H}}{1 - \vertiii{A^{-1}H}}\,\vertiii{A^{-1}} \leq 2\vertiii{A^{-1}}^2\vertiii H \to 0$ : $\Phi$ is continuous at every $A \in GL_n(\R)$ .

5. $(A+H)^2 = A^2 + AH + HA + H^2$ : the map $H \mapsto AH + HA$ is linear and the error $H^2$ is $O(\vertiii H^2)$ . Expanding $(A + H)^k$ and sorting by the number of $H$ -factors: the linear part is $\sum_{i=0}^{k-1}A^iHA^{k-1-i}$ , and the terms with $\geq 2$ factors $H$ are bounded by $\binom k2$ -many products of norm $\leq \vertiii A^{k-2}\vertiii H^2$ -scale: $O(\vertiii H^2)$ . One cannot collapse the sum to $kA^{k-1}H$ because $H$ and $A$ need not commute — the sum is the correct noncommutative derivative.

6. For small $H$ :

(A+H)^{-1} = (I + A^{-1}H)^{-1}A^{-1} = \bigl(I - A^{-1}H + O(\vertiii H^2)\bigr)A^{-1} = A^{-1} - A^{-1}HA^{-1} + O(\vertiii H^2) :

$\dd\Phi_A(H) = -A^{-1}HA^{-1}$ , linear in $H$ . For $n = 1$ : $\dd(1/a)(h) = -h/a^2$ , the familiar derivative.

7. Chain rule along the curve: $\bigl(A(t)^{-1}\bigr)' = \dd\Phi_{A(t)}(A'(t)) = -A(t)^{-1}A'(t)A(t)^{-1}$ . At $A(t) = I + tB$ , $t = 0$ : $(I + tB)^{-1} = I - tB + O(t^2)$ .

8. By question 5 and cyclic invariance of the trace:

\dd f_A(H) = \operatorname{tr}\Bigl(\sum_{i=0}^{k-1} A^iHA^{k-1-i}\Bigr) = k\operatorname{tr}\bigl(A^{k-1}H\bigr) .

Against the Frobenius product, $\dd f_A(H) = \operatorname{tr}\bigl((\nabla f)^{\mathsf T}H\bigr)$ requires $(\nabla f)^{\mathsf T} = kA^{k-1}$ : $\nabla f(A) = k\,(A^{k-1})^{\mathsf T}$ .

9. $f(A + H) - f(A) = 2\operatorname{tr} (A^{\mathsf T}H) + \operatorname{tr}(H^{\mathsf T}H)$ : the differential is $H \mapsto 2\operatorname{tr}(A^{\mathsf T}H) = 2\langle A, H\rangle$ , so $\nabla f(A) = 2A$ ; the second-order term is exactly $\norm H_F^2$ : the Hessian is twice the identity quadratic form, positive definite and constant, so $\norm\cdot_F^2$ is strictly convex (the Taylor formula is exact here).

10. By multilinearity in the columns, $\det(I + H) = \sum_{S\subseteq\{1,\dots,n\}}\det(M_S)$ where $M_S$ has column $h_j$ for $j \in S$ and $e_j$ otherwise. $S = \varnothing$ gives $1$ ; $S = \{j\}$ gives the determinant of $I$ with column $j$ replaced by $h_j$ , namely its $j$ -th entry $h_{jj}$ , summing to $\operatorname{tr}H$ ; each term with $\abs S \geq 2$ is a determinant with at least two columns of size $O(\vertiii H)$ , hence $O(\vertiii H^2)$ (multilinear maps on a finite-dimensional space are bounded), and there are finitely many. So $\det(I + H) = 1 + \operatorname{tr}H + O(\vertiii H^2)$ : $\dd(\det)_I = \operatorname{tr}$ .

11. $\det(A + H) = \det A\,\det(I + A^{-1}H) = \det A\,\bigl(1 + \operatorname{tr}(A^{-1}H) + O(\vertiii H^2)\bigr)$ : the differential is $H \mapsto \det(A)\operatorname{tr}(A^{-1}H)$ .

12. Laplace expansion along row $i$ : $\det A = \sum_j a_{ij}C_{ij}$ , and the cofactors $C_{ij}$ do not involve row $i$ : $\frac{\partial\det}{\partial a_{ij}} = C_{ij}$ . Hence

\dd(\det)_A(H) = \sum_{i,j}C_{ij}h_{ij} = \operatorname{tr}\bigl(\operatorname{com}(A)^{\mathsf T} H\bigr) = \operatorname{tr}\bigl(\operatorname{adj}(A)H\bigr), \qquad \nabla(\det)(A) = \operatorname{com}(A) .

For invertible $A$ , $\operatorname{adj}A = \det(A)A^{-1}$ recovers question 11. Along a $C^1$ curve, the chain rule reads $(\det A(t))' = \operatorname{tr}(\operatorname{adj} (A(t))\,A'(t))$ : Jacobi’s formula.

13. With $A' = MA$ and $\operatorname{adj}(A)A = \det(A)I$ :

(\det A)' = \operatorname{tr}\bigl(\operatorname{adj}(A)MA \bigr) = \operatorname{tr}\bigl(A\operatorname{adj}(A)M\bigr) = \det A\;\operatorname{tr}M

(cyclicity; $A\operatorname{adj}A = \det(A) I$ as well). The scalar linear ODE $y' = \operatorname{tr}(M(t))\,y$ has the unique solution $y(t) = y(0)\exp\bigl(\int_0^t \operatorname{tr}M\bigr)$ (Year 1): Liouville’s formula.

14. At $A \in SL_n(\R)$ , take $H = \frac1nA$ : $\dd(\det)_A\bigl(\tfrac1nA\bigr) = \frac1n\det(A) \operatorname{tr}(A^{-1}A) = \frac1n\cdot1\cdot n = 1 \neq 0$ : the differential is a nonzero linear form, hence surjective onto $\R$ at every point of the level set: $SL_n(\R)$ is a smooth level set (of dimension $n^2 - 1$ ).

15. $\sum_k\vertiii{A^k/k!} \leq \sum\vertiii A^k/k! = \eu^{\vertiii A}$ : absolute convergence (question 2’s completeness argument), with normal convergence on every ball $\vertiii A \leq R$ (bound $R^k/k!$ independent of $A$ ). Each partial sum is continuous (polynomial); the uniform limit on balls is continuous: $A \mapsto \eu^A$ is continuous, with $\vertiii{\eu^A} \leq \eu^{\vertiii A}$ .

16. Both series converge absolutely, so the Cauchy product is legitimate (Theorem 7.14):

\eu^A\eu^B = \sum_{n\geq0}\frac{1}{n!}\sum_{k=0}^n\binom nkA^kB^{n-k} = \sum_{n\geq0}\frac{(A+B)^n}{n!} = \eu^{A+B},

the binomial identity requiring $AB = BA$ . With $B = -A$ : $\eu^A\eu^{-A} = \eu^0 = I$ : every $\eu^A \in GL_n(\R)$ .

17. The series $\sum t^kA^k/k!$ and its derived series $\sum t^{k-1}A^k/(k-1)! = A\sum t^{k-1}A^{k-1}/(k-1)!$ converge normally on every segment $\abs t \leq T$ (bounds $T^k\vertiii A^k/k!$ ): the differentiation theorem for series (Theorem 10.11, applied entry by entry) gives $\frac{\dd}{\dd t}\eu^{tA} = A\eu^{tA}$ ; factoring $A$ on the right instead gives $\eu^{tA}A$ . Iterating: $C^\infty$ .

18. $A(t) = \eu^{tA}$ satisfies $A'(t) = A\,A(t)$ : Liouville’s formula (question 13) with constant $M = A$ gives $\det\eu^{tA} = \eu^{t\operatorname{tr}A}$ (value $1$ at $t = 0$ ); at $t = 1$ , $\det\eu^A = \eu^{\operatorname{tr}A}$ . Checks: $n = 1$ is the exponential itself; a nilpotent $A$ has $\operatorname{tr}A = 0$ and $\eu^A$ unipotent of determinant $1$ ; $\operatorname{tr}A = 0$ gives $\det\eu^A = 1$ : the trace-zero matrices are sent into $SL_n(\R)$ .

19. $\eu^H - I - H = \sum_{k\geq2}H^k/k!$ , of norm $\leq \vertiii H^2\eu^{\vertiii H} = O(\vertiii H^2)$ : $\dd(\exp)_0 = \mathrm{id}$ , invertible. Moreover $\exp$ is $C^1$ : by question 5, the differential candidate $H \mapsto \sum_k\frac1{k!}\sum_iA^iHA^{k-1-i}$ is a normally convergent series of linear maps depending continuously on $A$ (bounds $\vertiii A^{k-1}/(k-1)!$ on balls), so the partials exist and are continuous (Theorem 15.2 and the series transfer theorem). The inverse function theorem (Theorem 15.14) applies at $0$ : $\exp$ is a $C^1$ diffeomorphism of a neighborhood of $0$ onto a neighborhood of $I$ — matrices near $I$ have logarithms.

20. For symmetric $S = PDP^{\mathsf T}$ (spectral theorem): $\eu^S = P\eu^DP^{\mathsf T}$ is symmetric with eigenvalues $\eu^{\lambda_i} > 0$ : positive definite. Surjectivity: a positive definite $Q = P\operatorname{diag}(\mu_i)P^{\mathsf T}$ ( $\mu_i > 0$ ) is $\eu^S$ for $S = P\operatorname{diag}(\ln\mu_i)P^{\mathsf T}$ . Injectivity: $\eu^S$ determines its eigenspaces, which are exactly those of $S$ (on each eigenspace of $S$ for $\lambda$ , $\eu^S$ acts as $\eu^\lambda$ ; distinct $\lambda$ give distinct $\eu^\lambda$ ), and taking $\ln$ of the eigenvalues recovers $S$ . So $\exp$ is a bijection from symmetric matrices onto positive definite ones.

21. $F(A + H) = A^{\mathsf T}A + A^{\mathsf T}H + H^{\mathsf T}A + H^{\mathsf T}H$ : $\dd F_A(H) = A^{\mathsf T}H + H^{\mathsf T}A$ (values in $S_n$ ; error $O(\vertiii H^2)$ ). At $A \in O_n$ and for $S \in S_n$ , the choice $H = \frac12AS$ gives

A^{\mathsf T}\cdot\tfrac12AS + \tfrac12(AS)^{\mathsf T}A = \tfrac12 S + \tfrac12 S^{\mathsf T} = S :

surjective. $O_n = F^{-1}(I)$ is a smooth level set of dimension $n^2 - \dim S_n = \frac{n(n-1)}2$ .

22. Differentiating $A(t)^{\mathsf T}A(t) = I$ at $t = 0$ (with $A(0) = I$ ): $A'(0)^{\mathsf T} + A'(0) = 0$ : antisymmetric. Conversely, for $K^{\mathsf T} = -K$ : $(\eu^{tK})^{\mathsf T}\eu^{tK} = \eu^{tK^{\mathsf T}} \eu^{tK} = \eu^{-tK}\eu^{tK} = I$ (transpose the series termwise; the exponents commute): the curve stays in $O_n$ , with velocity $K$ at $t = 0$ . Tangent space at $I$ $=$ the antisymmetric matrices, of the expected dimension $\frac{n(n-1)}2$ .

23. $\operatorname{tr}K = 0$ for antisymmetric $K$ , so $\det\eu^K = \eu^0 = 1$ (question 18): the exponential lands in $SO_n$ . For $n = 2$ : $J^2 = -I$ , so $J^{2m} = (-1)^mI$ , $J^{2m+1} = (-1)^mJ$ , and

\eu^{\theta J} = \Bigl(\sum_m\frac{(-1)^m\theta^{2m}}{(2m)!}\Bigr)I + \Bigl(\sum_m\frac{(-1)^m\theta^{2m+1}}{(2m+1)!}\Bigr)J = \cos\theta\,I + \sin\theta\,J ,

the rotation by $\theta$ : the sine and cosine series live inside the matrix exponential.

24. For invertible $A, B$ : $\operatorname{adj}(AB) = \det(AB)(AB)^{-1} = \det(B)\det(A)B^{-1}A^{-1} = \operatorname{adj}(B)\operatorname{adj}(A)$ . Both sides of the identity are polynomial (hence continuous) maps of the entries of $(A, B)$ ; they agree on the dense subset $GL_n\times GL_n$ of $\mathcal M_n\times\mathcal M_n$ (question 3: approximate each factor), so they agree everywhere.

25. (i) Part I ran on completeness of finite-dimensional normed spaces (absolutely convergent series converge), Part IV on the same plus the spectral theorem for question 20, Part V’s local logarithm on the inverse function theorem. (ii) The differential-equations chapter leans on Liouville’s formula (question 13) for the Wronskian of linear systems, and on $\frac{\dd}{\dd t}\eu^{tA} = A\eu^{tA}$ (question 17), which is the statement that $\eu^{tA}$ solves $X' = AX$ . (iii) $\dd(\det)_I = \operatorname{tr}$ says the trace is the infinitesimal rate of volume change, and $\det\eu^A = \eu^{\operatorname{tr}A}$ integrates that statement globally. (iv) The Year 3 volume names the structures: $O_n$ and $SL_n(\R)$ are Lie groups, their tangent spaces at $I$ (antisymmetric and trace-zero matrices) are Lie algebras, and $\exp$ is the bridge between them.