---
title: "Functions of Two Variables"
book: "University Mathematics — Year 1"
subject: math
language: en
chapter: 25
exercises: 12
source: https://one-course.com/books/math/3/en/chapter/25-functions-of-two-variables
---

# Chapter 25 — Functions of Two Variables

The year ends with a first walk into higher dimension: functions $f(x, y)$ of two real variables. Everything generalizes — limits, [continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous), [derivatives](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def), extrema — but each notion gains a twist: limits can be approached along every direction at once, [derivatives](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def) split into partials, and the [gradient](#def-b1-multivar-partial) points the way uphill. The full theory (differentials, general $\R^n$, submanifolds) belongs to the second year; here we set the vocabulary and the first honest theorems.

## 25.1 The plane $\R^2$; continuity

**Definition 25.1.**

On $\R^2$, use the Euclidean norm $\norm{(x,y)} = \sqrt{x^2 + y^2}$ ([Chapter 23](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#ch-b1-euclid)). [Open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) balls, [neighborhoods](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open), *[open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) subsets* of $\R^2$ are defined exactly as in [Chapter 12](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#ch-b1-topology), with balls in place of [intervals](https://one-course.com/books/math/3/en/chapter/10-real-numbers#prop-b1-reals-intervals). A function $f \colon U \to \R$ ($U \subseteq \R^2$ [open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open)) is *[continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) at* $a \in U$ when

$$
\forall\varepsilon > 0,\ \exists\delta > 0, \quad
\norm{X - a} \leq \delta \implies \abs{f(X) - f(a)} \leq
\varepsilon,
$$

with the same sequential characterization as in one variable. Sums, products, quotients and compositions with [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) one-variable functions preserve [continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous); the coordinate [maps](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-map) are [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous), hence so are [polynomials](https://one-course.com/books/math/3/en/chapter/8-polynomials#def-b1-poly-def) in $(x,y)$.

**Example 25.2 (The polar bound, the clean way to prove a limit).**

Show that $f(x, y) = \dfrac{x^2y^2}{x^2 + y^2}$ (with $f(0,0) =
0$) is [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) at the origin. In polar [coordinates](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#prop-b1-vspaces-coordinates) $x =
\rho\cos\theta$, $y = \rho\sin\theta$:

$$
\abs{f} = \frac{\rho^4\cos^2\theta\sin^2\theta}{\rho^2}
= \rho^2\,(\cos\theta\sin\theta)^2 \leq \frac{\rho^2}{4}
\xrightarrow[\rho \to 0]{} 0 ,
$$

a bound *independent of $\theta$*: whatever the direction of approach, the values are squeezed to $0$. That uniformity in $\theta$ is the whole point — a bound like $\abs g =
\abs{\cos\theta\sin\theta}$ (no $\rho$ left) proves nothing, and indeed that $g$ is the discontinuous radial trap of the next example.

**Example 25.3 (The radial trap).**

Let $f(x, y) = \dfrac{xy}{x^2 + y^2}$ for $(x,y) \neq (0,0)$, $f(0,
0) = 0$. Along each axis, $f = 0 \to 0$; but along the diagonal $y
= x$, $f(x, x) = \frac12 \not\to 0$. *No limit at the origin*: approaching along every line, and even finding the same limit along each, is not enough (here the line limits disagree; worse examples agree along all lines yet fail along a parabola, [Exercise 25.3](#exo-b1-multivar-3)). [Continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) in each variable separately does not imply [continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous).

## 25.2 Partial derivatives

**Definition 25.4.**

The *partial derivatives* of $f$ at $(a, b)$ are the one-variable [derivatives](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def) along the axes:

$$
\frac{\partial f}{\partial x}(a,b)
= \lim_{h \to 0} \frac{f(a + h, b) - f(a,b)}{h},
\qquad
\frac{\partial f}{\partial y}(a,b)
= \lim_{k \to 0} \frac{f(a, b + k) - f(a,b)}{k}.
$$

$f$ is of class $C^1$ on $U$ when both exist and are [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) on $U$. The *gradient* is $\nabla f(a,b) =
\bigl(\frac{\partial f}{\partial x},\, \frac{\partial f}{\partial
y}\bigr)(a,b)$.

**Theorem 25.5 (C1C^1C1 implies a tangent plane).**

Let $f$ be $C^1$ on $U$ and $(a,b) \in U$. Then, as $(h, k) \to
(0,0)$:

$$
f(a + h, b + k) = f(a, b)
+ h\,\frac{\partial f}{\partial x}(a,b)
+ k\,\frac{\partial f}{\partial y}(a,b)
+ o\bigl(\norm{(h,k)}\bigr) .
$$

In particular $f$ is [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous), and the graph $z = f(x,y)$ has at each point the tangent plane read off the formula.

**Example 25.6 (Linear approximation in action).**

Estimate $f(1.02,\ 0.99)$ for $f(x, y) = x^3y^2$. At $(1, 1)$: $f = 1$, $\frac{\partial f}{\partial x} = 3x^2y^2 = 3$, $\frac{\partial f}{\partial y} = 2x^3y = 2$, so [Theorem 25.5](#thm-b1-multivar-c1) gives

$$
f(1.02,\ 0.99) \approx 1 + 3\,(0.02) + 2\,(-0.01) = 1.04 ,
$$

against the true value $1.02^3 \times 0.99^2 = 1.04006\dots$ — the error is of second order in the increments, as the $o(\norm{(h,k)})$ promises. The tangent plane to the graph at $(1, 1, 1)$ is $z = 1 + 3(x - 1) + 2(y - 1)$, the equation implicit in the estimate.

**Proof.** Move one coordinate at a time:

$$
f(a+h, b+k) - f(a,b)
= \bigl[f(a+h, b+k) - f(a, b+k)\bigr] + \bigl[f(a, b+k) -
f(a,b)\bigr].
$$

By the one-variable mean value theorem ([Theorem 14.9](https://one-course.com/books/math/3/en/chapter/14-differentiation#thm-b1-derivative-mvt)), the first bracket is $h\,
\frac{\partial f}{\partial x}(a + \theta h,\, b + k)$ for some $\theta \in \intoo{0}{1}$, and the second is $k\,\frac{\partial
f}{\partial y}(a,\, b + \theta' k)$. [Continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) of the partials at $(a,b)$ lets us write each as (value at $(a,b)$) $+$ (error $\to 0$); the total error is $h\,\varepsilon_1 +
k\,\varepsilon_2 = o(\norm{(h,k)})$ since $\abs h, \abs k \leq
\norm{(h,k)}$. ∎

**Theorem 25.7 (Chain rule).**

Let $f$ be $C^1$ on $U$ and $t \mapsto (x(t), y(t))$ be $C^1$ from an [interval](https://one-course.com/books/math/3/en/chapter/10-real-numbers#prop-b1-reals-intervals) into $U$. Then $g(t) = f\bigl(x(t), y(t)\bigr)$ is $C^1$, with

$$
g'(t) = x'(t)\,\frac{\partial f}{\partial x}\bigl(x(t),y(t)\bigr)
+ y'(t)\,\frac{\partial f}{\partial y}\bigl(x(t),y(t)\bigr)
= \bigl\langle \nabla f,\ (x', y')\bigr\rangle .
$$

**Proof.** Apply [Theorem 25.5](#thm-b1-multivar-c1) at $(x(t), y(t))$ with $(h, k) =
(x(t+s) - x(t),\, y(t+s) - y(t))$: as $s \to 0$, one-variable differentiability gives $h = s\,x'(t) + o(s)$ and $k = s\,y'(t)
+ o(s)$, so $\norm{(h, k)} = O(s)$ and

$$
g(t+s) - g(t)
= h\,\frac{\partial f}{\partial x} + k\,\frac{\partial
f}{\partial y} + o\bigl(\norm{(h,k)}\bigr)
= s\,\Bigl(x'\,\frac{\partial f}{\partial x} +
y'\,\frac{\partial f}{\partial y}\Bigr) + o(s) ,
$$

the final error absorbing both the $o(s)$’s of $h$ and $k$ (multiplied by the fixed values of the partials) and the $o(O(s))$ of the tangent-plane estimate. Divide by $s$ and let $s \to 0$. [Continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) of $g'$ follows from that of all ingredients. ∎

**Example 25.8 (The chain rule, verified both ways).**

Let $f(x, y) = x^2 y$ and $g(t) = f(t, t^2)$. Directly: $g(t) =
t^2\cdot t^2 = t^4$, so $g'(t) = 4t^3$. By the chain rule: $\frac{\partial f}{\partial x} = 2xy$ and $\frac{\partial
f}{\partial y} = x^2$, evaluated along the curve $(t, t^2)$:

$$
g'(t) = 1\cdot 2t\cdot t^2 + 2t\cdot t^2 = 2t^3 + 2t^3 = 4t^3 .
$$

The two computations agree, and the split is meaningful: $2t^3$ of the growth comes from moving *rightward* through the $x$-slope, $2t^3$ from moving *upward* through the $y$-slope. On curves where no closed form for $g$ exists, only the second computation survives — that is the theorem’s point.

**Remark 25.9 (Reading the gradient).**

Along a unit direction $u$, the chain rule on $t \mapsto f(a + tu)$ gives the *directional [derivative](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def)* $\langle \nabla f(a),
u\rangle$: maximal when $u$ points along $\nabla f(a)$ (by Cauchy–Schwarz, [Theorem 23.4](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-cs)). The [gradient](#def-b1-multivar-partial) is the direction of steepest ascent, and it is [orthogonal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) to the level curves $\{f = c\}$ (differentiate $f$ along a curve drawn in a level [set](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-sets): the chain rule gives $\langle\nabla f,\,
\text{tangent}\rangle = 0$).

**Example 25.10 (Level curves and gradients, on one function).**

Take $f(x, y) = x^2 - y^2$. Its level [sets](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-sets): $\{f = c\}$ is a hyperbola opening left-right for $c > 0$, up-down for $c < 0$, and the crossed pair of lines $y = \pm x$ for $c = 0$ — the contour [map](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-map) of a mountain pass, with the [saddle point](#met-b1-multivar-monge) at the origin where the two zero-level lines cross. [Gradient](#def-b1-multivar-partial): $\nabla
f = (2x, -2y)$. At the point $(2, 1)$ (on the level $c = 3$): $\nabla f = (4, -2)$, while the tangent vector of the level curve, parametrized near that point by $\bigl(t,
\sqrt{t^2 - 3}\bigr)$, is $\bigl(1, \frac{t}{\sqrt{t^2 -
3}}\bigr) = (1, 2)$ at $t = 2$ — and indeed

$$
\langle (4, -2),\ (1, 2)\rangle = 4 - 4 = 0 :
$$

[gradient](#def-b1-multivar-partial) perpendicular to the contour, pointing toward higher values of $f$ (here: away from the $y$-axis). Two more readings: the [gradient](#def-b1-multivar-partial) vanishes exactly at the saddle, where the contour [map](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-map) pinches; and the tangent *line* to the level curve at $(2,1)$ is $4(x - 2) - 2(y - 1) = 0$, i.e. $2x -
y = 3$ — the equation “$\langle \nabla f,\ M - M_0\rangle =
0$” that generalizes the ellipse tangent of [Exercise 24.11](https://one-course.com/books/math/3/en/chapter/24-plane-curves#exo-b1-curves-11).

**Theorem 25.11 (Schwarz).**

If $f$ is of class $C^2$ (partials of partials exist and are [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous)), then

$$
\frac{\partial^2 f}{\partial x\,\partial y}
= \frac{\partial^2 f}{\partial y\,\partial x} .
$$

**Proof.** *Admitted at this level.* ∎

## 25.3 Local extrema

**Method 25.12 (Extremum studies, organized).**

1. *Solve $\nabla f = 0$ completely.* Factor each partial whenever possible (products of [linear](https://one-course.com/books/math/3/en/chapter/20-linear-maps#def-b1-linmaps-def) factors split the system into transparent cases, as in [Example 25.16](#ex-b1-multivar-fourpoints) below); a forgotten case is a forgotten [critical point](#thm-b1-multivar-critical) .
2. *Classify each point* with the Monge data $r, s,  t$ — recomputed *at each point* , never once for all.
3. *If $rt - s^2 = 0$* , examine $f$ directly along well-chosen curves through the point (lines first, then parabolas), hunting either for two signs (no extremum) or for a locked sign with an argument covering all directions.
4. *Step back for the global picture* : check the behavior at infinity (a local minimum may be no global one), and if the domain is not [open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) , treat its [boundary](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closure) separately ( [Exercise 25.12](#exo-b1-multivar-12) ) — the [critical point](#thm-b1-multivar-critical) theorem sees [interior](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closure) points only.

**Theorem 25.13 (Critical points).**

If $f$ ($C^1$ on the [open set](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) $U$) has a local extremum at $(a,b)
\in U$, then $\nabla f(a,b) = (0,0)$: the point is *critical*.

**Proof.** The one-variable functions $x \mapsto f(x, b)$ and $y \mapsto f(a,
y)$ have [interior](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closure) local extrema at $a$, resp. $b$: [Proposition 14.7](https://one-course.com/books/math/3/en/chapter/14-differentiation#prop-b1-derivative-fermat) kills both partials. ∎

**Method 25.14 (Second-order test (Monge notation)).**

At a [critical point](#thm-b1-multivar-critical) of a $C^2$ function, set

$$
r = \frac{\partial^2 f}{\partial x^2},
\qquad
s = \frac{\partial^2 f}{\partial x \partial y},
\qquad
t = \frac{\partial^2 f}{\partial y^2}
\qquad (\text{values at the point}).
$$

- If $rt - s^2 > 0$ : local extremum — minimum for $r > 0$ , maximum for $r < 0$ ;
- if $rt - s^2 < 0$ : no extremum (a *saddle point* );
- if $rt - s^2 = 0$ : the test is silent; examine directly.

(Justification — a Taylor–Young expansion at order $2$ and the sign study of the quadratic form $r h^2 + 2shk + tk^2$ — is carried out in the second year; the test is used here as a working tool.)

**Example 25.15.**

$f(x,y) = x^3 + y^3 - 3xy$. [Critical points](#thm-b1-multivar-critical): $\nabla f = (3x^2 -
3y,\; 3y^2 - 3x) = 0$ gives $y = x^2$ and $x = y^2$, so $x = x^4$: $x \in \{0, 1\}$: points $(0,0)$ and $(1,1)$.

Second [derivatives](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def): $r = 6x$, $s = -3$, $t = 6y$. At $(0,0)$: $rt -
s^2 = -9 < 0$: saddle. At $(1,1)$: $rt - s^2 = 36 - 9 > 0$, $r = 6
> 0$: local minimum, $f(1,1) = -1$. (Not global: $f(x, 0) = x^3 \to
-\infty$.)

**Example 25.16 (A four-point study, in full).**

$f(x, y) = xy\,(3 - x - y) = 3xy - x^2y - xy^2$. [Gradient](#def-b1-multivar-partial):

$$
\frac{\partial f}{\partial x} = y\,(3 - 2x - y),
\qquad
\frac{\partial f}{\partial y} = x\,(3 - x - 2y).
$$

[Critical points](#thm-b1-multivar-critical): if $y = 0$, the second equation gives $x \in
\{0, 3\}$; if $x = 0$, the first gives $y \in \{0, 3\}$; if $xy
\neq 0$, solve $2x + y = 3$, $x + 2y = 3$: $x = y = 1$. Four points: $(0,0)$, $(3,0)$, $(0,3)$, $(1,1)$. Second [derivatives](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def): $r = -2y$, $s = 3 - 2x - 2y$, $t = -2x$.

- $(1,1)$ : $r = -2$ , $s = -1$ , $t = -2$ : $rt - s^2 = 3 >  0$ , $r < 0$ : local maximum, $f(1,1) = 1$ .
- $(0,0)$ : $r = t = 0$ , $s = 3$ : $rt - s^2 = -9 < 0$ : saddle; likewise $(3, 0)$ ( $s = -3$ ) and $(0, 3)$ : three saddles.

The maximum is only local: $f(-T, -T) = T^2(3 + 2T) \to
+\infty$. Symmetry check: $f(x, y) = f(y, x)$, and indeed the critical [set](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-sets) and the classification are symmetric in $x
\leftrightarrow y$. Interpretation: among rectangles-with-slack $x, y \geq 0$, $x + y \leq 3$, the product $xy(3 - x - y)$ of the three “parts” of $3$ is largest when the parts are equal — a two-variable shadow of the AM–GM principle.

![The two curves f = 0 for f = x3 + y3 - 3xy intersect at the critical points (0,0) (saddle) and (1,1) (local minimum).](https://one-course.com/images/onecourse/chapters/math-3/b1-multivar/fig-7abd314e582c.svg)

*The two curves $\nabla f = 0$ for $f = x^3 + y^3 - 3xy$ intersect at the [critical points](#thm-b1-multivar-critical) $(0,0)$ (saddle) and $(1,1)$ (local minimum).*

**Remark 25.17 (Common pitfalls).**

*[Partial derivatives](#def-b1-multivar-partial) may exist at a point of discontinuity*: the radial trap $g(x,y) = \frac{xy}{x^2+y^2}$ of [Example 25.3](#ex-b1-multivar-trap) has $\frac{\partial g}{\partial
x}(0,0) = \frac{\partial g}{\partial y}(0,0) = 0$ (both axis restrictions vanish identically), yet $g$ has no limit at the origin — partials probe two directions only, [continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) needs all of them; only the $C^1$ hypothesis restores order ([Theorem 25.5](#thm-b1-multivar-c1)). *Line limits never suffice*: [Exercise 25.3](#exo-b1-multivar-3)’s function has limit $0$ along every line and still no limit — always try parabolas (or polar bounds valid uniformly in $\theta$). *Critical is necessary, not sufficient*: saddles abound (three of four points in [Example 25.16](#ex-b1-multivar-fourpoints)); and the theorem holds on *[open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open)* [sets](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-sets) only — on a [closed](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closed) disc, extrema may sit on the [boundary](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closure) with nonzero [gradient](#def-b1-multivar-partial) ([Exercise 25.12](#exo-b1-multivar-12)). *The silent case $rt - s^2 = 0$ is genuinely silent*: $x^4
+ y^4$ (minimum) and $x^3 + y^3$ (neither) both have $r = s = t
= 0$ at the origin; only a direct sign study decides ([Exercise 25.6](#exo-b1-multivar-6), function $h$). *The [gradient](#def-b1-multivar-partial) is [orthogonal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) to level curves, not along them*: to follow a contour line, move perpendicular to $\nabla
f$; to climb fastest, move along it — mixing the two reverses the geometry of every contour [map](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-map).

**Remark 25.18 (Where two variables lead).**

This chapter is a doorway. The [gradient](#def-b1-multivar-partial) and the chain rule extend verbatim to $n$ variables in the Year 2 volume, where the $o(\norm{(h,k)})$ of [Theorem 25.5](#thm-b1-multivar-c1) becomes the *differential* and the Monge test is proved in full via the order-two Taylor formula and quadratic forms. The special case that can be settled *this* year — quadratic functions, for which the second-order expansion is exact — is the subject of the weekend problem, and it happens to be the case that runs the world’s data fitting: [least squares](#pb-b1-multivar-1) regression. Constrained extrema ([Exercise 25.5](#exo-b1-multivar-5) was a preview) become Lagrange multipliers in Year 2; harmonic functions ([Exercise 25.7](#exo-b1-multivar-7)) return in the complex analysis of the Year 3 volume.

**Remark 25.19 (Perspectives inside Book 3: the year, closed).**

This chapter is where the volume’s two halves shake hands. The analysis half supplied its tools one [derivative](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def) at a time: the mean value theorem drives [Theorem 25.5](#thm-b1-multivar-c1), Taylor expansions drive the extremum tests, and the $\varepsilon$’s of [Chapter 12](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#ch-b1-topology) came back with balls instead of [intervals](https://one-course.com/books/math/3/en/chapter/10-real-numbers#prop-b1-reals-intervals). The algebra half supplied the geometry: the [gradient](#def-b1-multivar-partial) is read through the [inner product](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-def) of [Chapter 23](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#ch-b1-euclid) (Cauchy–Schwarz makes it the steepest direction), the Monge data $(r, s, t)$ is a symmetric matrix of [Chapter 21](https://one-course.com/books/math/3/en/chapter/21-matrices#ch-b1-matrices) with the determinant test of [Chapter 22](https://one-course.com/books/math/3/en/chapter/22-determinants-and-linear-systems#ch-b1-det), and the weekend problem runs [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) on data vectors. Even the curves of [Chapter 24](https://one-course.com/books/math/3/en/chapter/24-plane-curves#ch-b1-curves) return as level [sets](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-sets). A reader who can reconstruct why each of these five hand-offs works has, in effect, revised the entire year — which is the real purpose of this final chapter.

![Least squares in one picture: four data points, the regression line y = 1.4x - 0.1, and the vertical residuals (dashed) whose squares the line minimizes — total 0.2, the smallest achievable. The weekend problem computes this line, proves it is the unique minimizer, and identifies the whole construction with an orthogonal projection in ℝ4.](https://one-course.com/images/onecourse/chapters/math-3/b1-multivar/fig-c07cb4f9038d.svg)

*[Least squares](#pb-b1-multivar-1) in one picture: four data points, the [regression line](#pb-b1-multivar-1) $y = 1.4x - 0.1$, and the vertical residuals (dashed) whose squares the line minimizes — total $0.2$, the smallest achievable. The weekend problem computes this line, proves it is the unique minimizer, and identifies the whole construction with an [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) in $\R^4$.*

## 25.4 Exercises

**Exercise 25.1 ★.**

Compute the [partial derivatives](#def-b1-multivar-partial): $f(x,y) = x^2 y + \eu^{xy}$; $\;g(x,y) = \ln(x^2 + y^2)$ (on $\R^2\setminus\{0\}$); $\;h(x,y) = \arctan\frac yx$ (on $x > 0$).

**Solution of Exercise 25.1.**

$\dfrac{\partial f}{\partial x} = 2xy + y\,\eu^{xy}$, $\dfrac{\partial f}{\partial y} = x^2 + x\,\eu^{xy}$.

$\dfrac{\partial g}{\partial x} = \dfrac{2x}{x^2+y^2}$, $\dfrac{\partial g}{\partial y} = \dfrac{2y}{x^2+y^2}$.

$\dfrac{\partial h}{\partial x} = \dfrac{-y/x^2}{1 + y^2/x^2} =
\dfrac{-y}{x^2+y^2}$, $\dfrac{\partial h}{\partial y} = \dfrac{x}{x^2+y^2}$.

**Exercise 25.2 ★.**

Study the [continuity](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) at $(0,0)$ (with value $0$ there) of:

$$
f(x,y) = \frac{x^2 y}{x^2 + y^2},
\qquad
g(x,y) = \frac{xy}{x^2 + y^2},
\qquad
h(x,y) = \frac{x^3 + y^3}{x^2 + y^2}.
$$

*(Polar [coordinates](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#prop-b1-vspaces-coordinates) $x = \rho\cos\theta$, $y = \rho\sin\theta$ help: bound by a function of $\rho$ alone when possible.)*

**Solution of Exercise 25.2.**

In polar [coordinates](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#prop-b1-vspaces-coordinates) ($\rho \to 0$):

$\abs{f} = \dfrac{\rho^3\abs{\cos^2\theta\sin\theta}}{\rho^2} \leq
\rho \to 0$: [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous).

$g = \cos\theta\sin\theta$: independent of $\rho$, taking different values along different rays (cf. [Example 25.3](#ex-b1-multivar-trap)): no limit, not [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous).

$\abs h \leq \dfrac{\rho^3(\abs{\cos^3\theta} +
\abs{\sin^3\theta})}{\rho^2} \leq 2\rho \to 0$: [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous).

**Exercise 25.3 ★★.**

Let $f(x,y) = \dfrac{x^2 y}{x^4 + y^2}$ ($f(0,0) = 0$). Prove that $f$ has limit $0$ at the origin *along every straight line*, but that $f\bigl(x, x^2\bigr) = \frac12$: $f$ is not [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) at $(0,0)$.

**Solution of Exercise 25.3.**

Along $y = mx$: $f(x, mx) = \dfrac{m x^3}{x^4 + m^2 x^2} =
\dfrac{mx}{x^2 + m^2} \to 0$ (for $m \neq 0$; along $y = 0$ and the $y$-axis, $f = 0$). So every straight-line limit is $0$. But on the parabola $y = x^2$:

$$
f(x, x^2) = \frac{x^4}{x^4 + x^4} = \frac12 :
$$

the sequence $\bigl(\frac1n, \frac{1}{n^2}\bigr) \to (0,0)$ has $f \to \frac12 \neq 0$. Not [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous): lines do not suffice to test two-variable limits.

**Exercise 25.4 ★.**

Verify Schwarz’s theorem by hand on $f(x, y) = x^3 y^2 + \sin(xy)$.

**Solution of Exercise 25.4.**

$\frac{\partial f}{\partial x} = 3x^2y^2 + y\cos(xy)$; then

$$
\frac{\partial^2 f}{\partial y\,\partial x}
= 6x^2 y + \cos(xy) - xy\sin(xy) .
$$

$\frac{\partial f}{\partial y} = 2x^3 y + x\cos(xy)$; then

$$
\frac{\partial^2 f}{\partial x\,\partial y}
= 6x^2 y + \cos(xy) - xy\sin(xy) :
$$

equal, as Schwarz promises.

**Exercise 25.5 ★★.**

Let $f$ be $C^1$ on $\R^2$ and $g(t) = f(\cos t, \sin t)$. Express $g'(t)$ via the chain rule. Deduce that $f$ restricted to the unit circle attains extrema at points where $\nabla f$ is parallel to the radius vector.

**Solution of Exercise 25.5.**

By [Theorem 25.7](#thm-b1-multivar-chain) with $(x, y) = (\cos t, \sin t)$:

$$
g'(t) = -\sin t\,\frac{\partial f}{\partial x}(\cos t, \sin t)
+ \cos t\,\frac{\partial f}{\partial y}(\cos t, \sin t)
= \bigl\langle \nabla f,\ (-\sin t, \cos t)\bigr\rangle .
$$

At an extremum of $g$, $g'(t) = 0$: $\nabla f$ is [orthogonal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) to the tangent vector $(-\sin t, \cos t)$ of the circle, hence parallel to the radius vector $(\cos t, \sin t)$ (the plane [orthogonal complement](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) of a unit vector is the line it [spans](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#def-b1-vspaces-span), taken perpendicular). This is the simplest instance of a Lagrange multiplier.

**Exercise 25.6 ★★.**

Find and classify the [critical points](#thm-b1-multivar-critical) of:

$$
f(x, y) = x^2 + xy + y^2 - 3x,
\qquad
g(x, y) = x^2 - y^2 + 4y,
\qquad
h(x, y) = x^4 + y^4 - 2(x - y)^2 .
$$

*(For $h$, the determinant test is silent at the origin: examine $h(x, x)$ and $h(x, -x)$.)*

**Solution of Exercise 25.6.**

$f$: $\nabla f = (2x + y - 3,\; x + 2y) = 0$: $y = -\frac x2$ and $2x - \frac x2 = 3$: $x = 2$, $y = -1$. Second order: $r = 2$, $s =
1$, $t = 2$: $rt - s^2 = 3 > 0$, $r > 0$: local (indeed global — quadratic) minimum at $(2, -1)$, value $f(2,-1) = -3$.

$g$: $\nabla g = (2x,\; -2y + 4) = 0$: point $(0, 2)$; $r = 2$, $s
= 0$, $t = -2$: $rt - s^2 = -4 < 0$: saddle.

$h$: $\nabla h = \bigl(4x^3 - 4(x - y),\; 4y^3 + 4(x-y)\bigr) = 0$. Adding: $x^3 + y^3 = 0$, so $y = -x$; substituting: $4x^3 - 8x = 0$: $x \in \{0, \pm\sqrt2\}$. [Critical points](#thm-b1-multivar-critical): $(0,0)$, $(\sqrt2, -\sqrt2)$, $(-\sqrt2, \sqrt2)$. At $(\pm\sqrt2, \mp\sqrt2)$: $r = 12x^2 - 4 = 20$, $s = 4$, $t =
20$: $rt - s^2 > 0$, $r > 0$: local minima (value $h = 4 + 4 - 16 =
-8$). At $(0,0)$: $r = t = -4$, $s = 4$: $rt - s^2 = 0$: silent. Examine: $h(x, x) = 2x^4 > 0$ and $h(x, -x) = 2x^4 - 8x^2 < 0$ for small $x \neq 0$: both signs in every [neighborhood](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) — no extremum at the origin.

**Exercise 25.7 ★★.**

A function $f$ is *harmonic* when $\frac{\partial^2 f}{\partial
x^2} + \frac{\partial^2 f}{\partial y^2} = 0$. Check that $x^2 -
y^2$, $xy$, $\eu^x\cos y$ and $\ln(x^2 + y^2)$ (off the origin) are harmonic.

**Solution of Exercise 25.7.**

$x^2 - y^2$: second partials $2$ and $-2$: sum $0$. $xy$: both pure second partials vanish. $\eu^x\cos y$: $\frac{\partial^2}{\partial
x^2} = \eu^x\cos y$, $\frac{\partial^2}{\partial y^2} =
-\eu^x\cos y$: sum $0$. $\ln(x^2+y^2)$: from [Exercise 25.1](#exo-b1-multivar-1),

$$
\frac{\partial^2}{\partial x^2}\ln(x^2+y^2)
= \frac{2(x^2+y^2) - 4x^2}{(x^2+y^2)^2}
= \frac{2(y^2 - x^2)}{(x^2+y^2)^2},
$$

and the $y$-version is its opposite: sum $0$.

**Exercise 25.8 ★★★.**

Find the point of the plane $\{x + 2y - z = 4\} \subseteq \R^3$ closest to the origin, two ways: by [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) ([Chapter 23](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#ch-b1-euclid)), and by minimizing the two-variable function $f(x, y) = x^2 + y^2 + (x + 2y - 4)^2$ obtained by eliminating $z$.

**Solution of Exercise 25.8.**

*[Projection](https://one-course.com/books/math/3/en/chapter/20-linear-maps#def-b1-linmaps-projection):* the plane $P$ has normal $n = (1, 2, -1)$ and passes through $A = (4, 0, 0)$. The closest point to the origin is $O$ projected: $p = O + \frac{\langle A - O, n\rangle}{\norm
n^2}\,n = \frac{4}{6}(1,2,-1) = \bigl(\frac23, \frac43,
-\frac23\bigr)$, at distance $\frac{4}{\sqrt 6}$. (Formula: distance from origin to plane $\langle X, n \rangle = c$ is $\frac{\abs c}{\norm n}$ with $c = 4$.)

*Minimization:* $f(x,y) = x^2 + y^2 + (x + 2y - 4)^2$. Writing $w = x + 2y - 4$ for the last factor:

$$
\nabla f = \bigl(2x + 2w,\; 2y + 4w\bigr) = 0
\iff x = -w \text{ and } y = -2w .
$$

Substituting into the definition of $w$: $w = -w - 4w - 4$, so $w =
-\frac23$, giving $x = \frac23$, $y = \frac43$, and $z = w =
-\frac23$. Same point as by [projection](https://one-course.com/books/math/3/en/chapter/20-linear-maps#def-b1-linmaps-projection), at distance $\sqrt{\frac49 + \frac{16}{9} + \frac49} = \frac{2\sqrt 6}{3} =
\frac{4}{\sqrt6}$; it is a minimum since $f \to \infty$ at infinity (a positive-definite quadratic plus [linear](https://one-course.com/books/math/3/en/chapter/20-linear-maps#def-b1-linmaps-def) terms).

**Exercise 25.9 ★★★.**

(Laplacian in polar [coordinates](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#prop-b1-vspaces-coordinates), first contact) Let $f$ be $C^2$ on $\R^2 \setminus \{0\}$ and *radial*: $f(x, y) =
\varphi\bigl(\sqrt{x^2+y^2}\bigr)$ with $\varphi$ of class $C^2$ on $\intoo{0}{+\infty}$. Prove that

$$
\frac{\partial^2 f}{\partial x^2} + \frac{\partial^2 f}{\partial
y^2} = \varphi''(\rho) + \frac{\varphi'(\rho)}{\rho},
\qquad \rho = \sqrt{x^2 + y^2},
$$

and find all radial harmonic functions on the punctured plane.

**Solution of Exercise 25.9.**

With $\rho = \sqrt{x^2+y^2}$: $\frac{\partial \rho}{\partial x} =
\frac{x}{\rho}$, so

$$
\frac{\partial f}{\partial x} = \varphi'(\rho)\,\frac{x}{\rho},
\qquad
\frac{\partial^2 f}{\partial x^2}
= \varphi''(\rho)\,\frac{x^2}{\rho^2}
+ \varphi'(\rho)\,\Bigl(\frac{1}{\rho} -
\frac{x^2}{\rho^3}\Bigr),
$$

(quotient and chain rules). Adding the symmetric $y$-expression: the $\varphi''$ terms collect $\frac{x^2 + y^2}{\rho^2} = 1$, the $\varphi'$ terms collect $\frac{2}{\rho} - \frac{x^2 +
y^2}{\rho^3} = \frac{1}{\rho}$:

$$
\Delta f = \varphi'' + \frac{\varphi'}{\rho} .
$$

Radial harmonic: solve $\varphi'' + \frac{\varphi'}{\rho} = 0$: the first-order equation $u' + \frac u\rho = 0$ for $u = \varphi'$ gives $u = \frac{c}{\rho}$ ([Theorem 5.2](https://one-course.com/books/math/3/en/chapter/5-linear-differential-equations#thm-b1-diffeq-homogeneous1)), then $\varphi = c\ln\rho + d$. The radial harmonic functions are $c\,\ln\sqrt{x^2 + y^2} + d$ — the logarithmic potential of [Exercise 25.7](#exo-b1-multivar-7) and the constants, nothing else.

**Exercise 25.10 ★★.**

Give the equation of the tangent plane to the graph $z = xy$ at the point $(1, 1, 1)$. Then find all points of the graph of $f(x,y) = x^3 + y^3 - 3xy$ where the tangent plane is *horizontal*, and relate the answer to [Example 25.15](#ex-b1-multivar-extrema).

**Solution of Exercise 25.10.**

For $z = xy$ at $(1,1)$: partials $y = 1$ and $x = 1$, tangent plane $z = 1 + (x - 1) + (y - 1) = x + y - 1$. Horizontal tangent plane means both partials vanish, i.e. $\nabla f = 0$: by [Example 25.15](#ex-b1-multivar-extrema), exactly the [critical points](#thm-b1-multivar-critical) $(0, 0)$ and $(1, 1)$, with horizontal planes $z = 0$ and $z =
-1$. “Horizontal tangent plane” and “[critical point](#thm-b1-multivar-critical)” are the same notion, seen on the graph and in the formula.

**Exercise 25.11 ★★.**

Let $f$ be $C^1$ on $\R^2$.

1. If $\frac{\partial f}{\partial x} = 0$ everywhere, prove that $f(x, y)$ depends only on $y$ .
2. Find all $C^1$ solutions of the equation $\frac{\partial f}{\partial x} = \frac{\partial  f}{\partial y}$ on $\R^2$ . *(Set $g(u, v) =  f\bigl(\frac{u+v}2, \frac{u-v}2\bigr)$ and compute $\frac{\partial g}{\partial v}$ by the chain rule.)*

**Solution of Exercise 25.11.**

1. For fixed $y$ , the one-variable function $x \mapsto  f(x,y)$ has zero [derivative](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def) on $\R$ , hence is constant ( [Theorem 14.9](https://one-course.com/books/math/3/en/chapter/14-differentiation#thm-b1-derivative-mvt) ): $f(x, y) = f(0, y)$ for all $x$ : $f$ depends only on $y$ .
2. Let $g(u, v) = f\bigl(\frac{u+v}2, \frac{u-v}2\bigr)$. By the chain rule ([Theorem 25.7](#thm-b1-multivar-chain), applied in the variable $v$ with $u$ frozen): $$\frac{\partial g}{\partial v}  = \frac12\,\frac{\partial f}{\partial x}  - \frac12\,\frac{\partial f}{\partial y} = 0 .$$ By (1), $g$ depends only on $u$: $g(u, v) = \varphi(u)$ with $\varphi$ of class $C^1$, and reversing the change of variables ($u = x + y$, $v = x - y$), $$f(x, y) = \varphi(x + y).$$ Conversely every such $f$ satisfies $f_x = f_y =  \varphi'$: the solutions are exactly the $C^1$ functions of $x + y$.

**Exercise 25.12 ★★★.**

Find the global maximum and minimum of $f(x, y) = xy$ on the [closed](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closed) disc $x^2 + y^2 \leq 1$. *(Admit the two-variable extreme value theorem: a [continuous](https://one-course.com/books/math/3/en/chapter/13-limits-and-continuity#def-b1-continuity-continuous) function on the [closed](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closed) disc attains its bounds — proved in the Year 2 volume. Treat the [open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) disc by [critical points](#thm-b1-multivar-critical) and the [boundary](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closure) circle by the parametrization of [Exercise 25.5](#exo-b1-multivar-5).)*

**Solution of Exercise 25.12.**

On the [open](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) disc, an extremum would be critical: $\nabla f = (y,
x) = 0$ only at $(0,0)$, where $f = 0$; it is a saddle ($f(\pm
\varepsilon, \pm\varepsilon) = \varepsilon^2 > 0 > -\varepsilon^2
= f(\pm\varepsilon, \mp\varepsilon)$), so no extremum there. By the admitted extreme value theorem the bounds are attained, necessarily on the [boundary](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-closure) circle. There, with [Exercise 25.5](#exo-b1-multivar-5),

$$
f(\cos t, \sin t) = \cos t\sin t = \frac{\sin 2t}{2}
\in \intcc{-\tfrac12}{\tfrac12},
$$

with maximum $\frac12$ at $t = \frac\pi4, \frac{5\pi}4$ (points $\pm\frac{1}{\sqrt2}(1,1)$) and minimum $-\frac12$ at $t =
\frac{3\pi}4, \frac{7\pi}4$ (points $\pm\frac{1}{\sqrt2}(1,-1)$). Global maximum $\frac12$, global minimum $-\frac12$.

## 25.5 Problem: least squares and the regression line

**Problem 25.1.**

Given $n$ data points, which straight line passes “closest” to all of them? Legendre and Gauss answered: the line minimizing the sum of the *squared* vertical errors — because that minimization is exactly solvable by linear algebra. This problem first proves the Monge test of [Method 25.14](#met-b1-multivar-monge) honestly for quadratic functions (the one case where the second-order expansion is exact), then builds the [normal equations](#pb-b1-multivar-1), the [regression line](#pb-b1-multivar-1), and the correlation coefficient on top of the Euclidean geometry of [Chapter 23](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#ch-b1-euclid).

**Part I — Quadratic functions: the Monge test, proved.** Fix reals $r, s, t$ and let $q(h, k) = r h^2 + 2s hk +
t k^2$.

1. Suppose $r \neq 0$. Establish the completed-square form $$q(h, k) = r\Bigl(h + \frac{s}{r}k\Bigr)^{\!2}  + \frac{rt - s^2}{r}\,k^2 ,$$ and deduce: if $rt - s^2 > 0$, $q$ has the strict sign of $r$ off the origin; if $rt - s^2 < 0$, $q$ takes both signs.
2. Settle the remaining cases: $r = 0$ , $t \neq 0$ (symmetric); and $r = t = 0$ ( $q = 2shk$ ). Conclude: $q$ takes both signs iff $rt - s^2 < 0$ , and $q$ vanishes only at the origin iff $rt - s^2 > 0$ .
3. Let now $f(x, y) = \frac12\bigl(r x^2 + 2s xy + t  y^2\bigr) + \beta x + \gamma y + \delta$ be a quadratic function with a [critical point](#thm-b1-multivar-critical) $X_0 = (a, b)$. Prove the *exact* expansion $$f(X_0 + (h,k)) = f(X_0) + \tfrac12\,q(h, k)  \qquad (\text{no remainder}),$$ and deduce the Monge classification for quadratic functions: strict global minimum if $rt - s^2 > 0$, $r >  0$; strict global maximum if $rt - s^2 > 0$, $r < 0$; saddle if $rt - s^2 < 0$.
4. Suppose $rt - s^2 > 0$ and $r > 0$ (so also $t > 0$). Prove the explicit coercivity bound $$q(h, k) \geq c\,(h^2 + k^2),  \qquad  c = \frac{rt - s^2}{r + t} > 0 .$$ *(Show that the quadratic form $q - c(h^2 + k^2)$, with coefficients $r - c$, $s$, $t - c$, still has nonnegative discriminant-test data.)*
5. Deduce that a quadratic function with positive definite quadratic part tends to $+\infty$ as $\norm{(x,y)} \to  \infty$ , and therefore has a *unique global* minimizer: its [critical point](#thm-b1-multivar-critical) . Contrast with [Example 25.15](#ex-b1-multivar-extrema) , where a local minimum of a nonquadratic function was not global.

**Part II — The [normal equations](#pb-b1-multivar-1).** Let $C_1,
C_2, b \in \R^n$ (canonical [inner product](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-def)), and

$$
f(u, v) = \norm{u\,C_1 + v\,C_2 - b}^2 .
$$

6. Expand $f$ and show it is a quadratic function of $(u,  v)$ with quadratic-part coefficients $r = 2\norm{C_1}^2$ , $s = 2\langle C_1, C_2\rangle$ , $t = 2\norm{C_2}^2$ .
7. Show that $rt - s^2 > 0$ if and only if $(C_1, C_2)$ is [free](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#def-b1-vspaces-free) *(the equality case of Cauchy–Schwarz, [Theorem 23.4](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-cs))* ; assume this from now on.
8. Show that the critical equations $\nabla f = 0$ are the *[normal equations](#pb-b1-multivar-1)* $$\begin{pmatrix}  \norm{C_1}^2 & \langle C_1, C_2\rangle\\  \langle C_1, C_2\rangle & \norm{C_2}^2  \end{pmatrix}  \begin{pmatrix} u\\ v\end{pmatrix}  =  \begin{pmatrix} \langle C_1, b\rangle\\ \langle C_2,  b\rangle\end{pmatrix}$$ — the Gram matrix of [Exercise 23.11](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#exo-b1-euclid-11) on the left — and that they say exactly: $b - (uC_1 + vC_2)  \perp C_1, C_2$. Conclude with Part I and [Theorem 23.10](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection): the unique minimizer gives $p = uC_1 + vC_2 =$ the [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) of $b$ onto $\operatorname{Vect}(C_1, C_2)$.
9. Show that the minimal value is $\norm{b}^2 - \langle p,  b\rangle = \norm{b - p}^2$ , and draw the Pythagorean picture.

**Part III — The [regression line](#pb-b1-multivar-1).** Data points $(x_1, y_1), \dots, (x_n, y_n)$, not all $x_i$ equal. Minimize

$$
E(\alpha, \beta) = \sum_{i=1}^{n}\bigl(y_i - \alpha - \beta
x_i\bigr)^2 .
$$

Write $\bar x = \frac1n\sum x_i$, $\bar y = \frac1n\sum y_i$, $v_x = \frac1n\sum x_i^2 - \bar x^2$, $v_y = \frac1n\sum y_i^2 -
\bar y^2$, $c_{xy} = \frac1n\sum x_i y_i - \bar x\bar y$.

10. Recognize Part II with $C_1 = (1, \dots, 1)$, $C_2 =  (x_1, \dots, x_n)$, $b = (y_1, \dots, y_n)$, and write the [normal equations](#pb-b1-multivar-1) $$n\alpha + \Bigl(\sum x_i\Bigr)\beta = \sum y_i,  \qquad  \Bigl(\sum x_i\Bigr)\alpha + \Bigl(\sum  x_i^2\Bigr)\beta = \sum x_i y_i .$$
11. Solve them: $$\beta = \frac{c_{xy}}{v_x},  \qquad  \alpha = \bar y - \beta\,\bar x ,$$ and observe that the *[regression line](#pb-b1-multivar-1)* $y = \alpha  + \beta x$ passes through the mean point $(\bar x, \bar  y)$.
12. Check $v_x > 0$ exactly when the $x_i$ are not all equal, and match this with question 7.
13. Prove that the minimal error is $$E_{\min} = n\Bigl(v_y - \frac{c_{xy}^2}{v_x}\Bigr)  = n\,v_y\,(1 - \rho^2),  \qquad  \rho = \frac{c_{xy}}{\sqrt{v_x v_y}}  \quad (v_y \neq 0).$$ Deduce $\abs\rho \leq 1$, with $\abs\rho = 1$ if and only if the data are perfectly aligned.
14. Worked example: for the data $(0,0)$ , $(1,1)$ , $(2,3)$ , $(3,4)$ , compute $\bar x, \bar y, v_x, c_{xy}$ , the [regression line](#pb-b1-multivar-1) , the four residuals and $E_{\min}$ .
15. Prove that the residuals $\varepsilon_i = y_i - \alpha -  \beta x_i$ of the optimal line satisfy $\sum_i  \varepsilon_i = 0$ and $\sum_i x_i\varepsilon_i = 0$ , and interpret both via orthogonality.

**Part IV — Variations and applications.**

16. (Best constant) Show that the constant $\alpha$ minimizing $\sum_i (y_i - \alpha)^2$ is the mean $\bar  y$ , and that the minimal value is $n\,v_y$ : the variance measures the failure of the data to be constant.
17. (Line through the origin) Show that the slope minimizing $\sum_i (y_i - \beta x_i)^2$ is $\beta_0 = \frac{\sum  x_iy_i}{\sum x_i^2}$ , and give a condition on the data for $\beta_0$ to coincide with the slope $c_{xy}/v_x$ of question 11.
18. (Distance to a line, again) For a point $M$ and the line $D$ through $P_0$ directed by the unit vector $w$ , minimize $g(\tau) = \norm{P_0 + \tau w - M}^2$ and deduce $d(M, D)^2 = \norm{M - P_0}^2 - \langle M - P_0,  w\rangle^2$ ; recover the formula $d = \frac{\abs{a x_0 +  b y_0 + c}}{\sqrt{a^2 + b^2}}$ for the line $ax + by + c  = 0$ in the plane.
19. (Analysis of variance) Prove the decomposition $v_y =  \beta^2 v_x + \frac{E_{\min}}n$ : the variance of the $y_i$ splits into the part explained by the line plus the residual variance.
20. (Parabolic fit) To fit $y = a + bx + cx^2$, show that the [normal equations](#pb-b1-multivar-1) are the $3 \times 3$ system with the moment matrix $$\begin{pmatrix}  n & \sum x_i & \sum x_i^2\\  \sum x_i & \sum x_i^2 & \sum x_i^3\\  \sum x_i^2 & \sum x_i^3 & \sum x_i^4  \end{pmatrix},$$ a Gram matrix which is invertible as soon as three of the $x_i$ are distinct *(freeness of $(1, X, X^2)$ sampled at the data, [Exercise 23.11](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#exo-b1-euclid-11); compare the moment matrices of the weekend problem of [Chapter 22](https://one-course.com/books/math/3/en/chapter/22-determinants-and-linear-systems#ch-b1-det))*.

**Part V — Robustness, and synthesis.**

21. Classify the [critical points](#thm-b1-multivar-critical) of the three quadratic functions $$x^2 + xy + y^2, \qquad x^2 + 3xy + y^2, \qquad  x^2 + 2xy + y^2 ,$$ by Part I, treating the degenerate third case by hand (where is the minimum attained?).
22. (Outlier experiment) Append the point $(10, 0)$ to the data of question 14 and recompute the slope $\beta$ . What happened, and why is the *squared* error so sensitive to one distant point?
23. (Weighted [least squares](#pb-b1-multivar-1) ) Given weights $w_i > 0$ , minimize $\sum_i w_i(y_i - \alpha - \beta x_i)^2$ : show the solution is given by the same formulas as question 11 with weighted means, variance and covariance (define them).
24. Show that for the optimal line, $E_{\min} = 0$ forces all points to lie exactly on it, and connect with the equality case $\abs\rho = 1$ of question 13; test on the aligned data $(1,1), (2,3), (3,5)$ .
25. Synthesis, in four sentences: why quadratic functions are the one class where this chapter’s second-order test needs no admitted theorem; how the [normal equations](#pb-b1-multivar-1) identify the analytic minimization with the [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) of [Chapter 23](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#ch-b1-euclid) ; what the correlation coefficient measures and which inequality bounds it; and which pieces of Chapters 18–23 (bases, Gram and moment matrices, [projections](https://one-course.com/books/math/3/en/chapter/20-linear-maps#def-b1-linmaps-projection) ) reappeared. Name the method and the equations studied in Parts II–III.

**Solution of Problem 25.1.**

**1.** Expanding the right-hand side:

$$
r\Bigl(h + \frac srk\Bigr)^{\!2} + \frac{rt - s^2}{r}k^2
= rh^2 + 2shk + \frac{s^2}{r}k^2 + \frac{rt - s^2}{r}k^2
= q(h,k) .
$$

If $rt - s^2 > 0$: both squares carry the factor sign of $r$, and $q(h,k) = 0$ forces $k = 0$ then $h = 0$: strict sign of $r$ off the origin. If $rt - s^2 < 0$: $q(1, 0) = r$ while $q\bigl(-\frac sr, 1\bigr) = \frac{rt - s^2}{r}$ has the opposite sign: both signs occur.

**2.** If $r = 0 \neq t$, exchange the roles of $h$ and $k$ (the completed square in $k$), with the same conclusions; note $rt - s^2 = -s^2 < 0$ as soon as $s \neq 0$, and indeed $q =
2shk + tk^2$ takes both signs then ($k$ small against $h$). If $r = t = 0$: $q = 2shk$, both signs iff $s \neq 0$, i.e. iff $rt - s^2 = -s^2 < 0$. Summary: $q$ takes both signs $\iff rt -
s^2 < 0$; $q$ vanishes only at the origin (definite) $\iff rt -
s^2 > 0$ (which forces $r \neq 0$, since $r = 0$ gives $q(1,0) =
0$).

**3.** With $f_x = rx + sy + \beta$ and $f_y = sx + ty +
\gamma$, direct expansion gives

$$
f(a + h, b + k) = f(a, b) + h\,f_x(a,b) + k\,f_y(a,b)
+ \tfrac12 q(h, k),
$$

with no higher terms (the function is a [polynomial](https://one-course.com/books/math/3/en/chapter/8-polynomials#def-b1-poly-def) of degree $2$). At a [critical point](#thm-b1-multivar-critical) the [linear](https://one-course.com/books/math/3/en/chapter/20-linear-maps#def-b1-linmaps-def) part vanishes: $f(X_0 + H)
- f(X_0) = \frac12 q(H)$ *exactly*, so the sign study of questions 1–2 classifies: strict global minimum ($rt - s^2 >
0$, $r > 0$), strict global maximum ($rt - s^2 > 0$, $r < 0$), saddle — both signs in every [neighborhood](https://one-course.com/books/math/3/en/chapter/12-topology-of-the-real-line#def-b1-topology-open) — when $rt - s^2 <
0$. This proves [Method 25.14](#met-b1-multivar-monge) for quadratic functions.

**4.** The form $q - c(h^2 + k^2)$ has data $r - c$, $s$, $t - c$. With $c = \frac{rt - s^2}{r + t}$ (note $t > 0$ since $rt > s^2 \geq 0$ and $r > 0$):

$$
(r - c)(t - c) - s^2 = rt - s^2 - c(r + t) + c^2 = c^2 \geq 0,
$$

and $r - c \geq 0$ ($c \leq r \iff rt - s^2 \leq r^2 + rt$, true). If $r - c > 0$, the completed square of question 1 shows $q - c(h^2 + k^2) \geq 0$; if $r - c = 0$, then $s = 0$ (from $-s^2 = -c^2 \leq \dots$ forcing the displayed quantity $\geq 0$ with first factor $0$) and the form is $(t - c)k^2 \geq
0$. In both cases $q(h,k) \geq c(h^2 + k^2)$.

**5.** By questions 3–4, $f(X) = f(X_0) + \frac12 q(X -
X_0) \geq f(X_0) + \frac c2\norm{X - X_0}^2 \to +\infty$ as $\norm X \to \infty$: $f$ is coercive, and the inequality is strict for $X \neq X_0$: the [critical point](#thm-b1-multivar-critical) is the unique global minimizer. For the cubic $f = x^3 + y^3 - 3xy$ of [Example 25.15](#ex-b1-multivar-extrema), no such conclusion holds: $f(x,0)
= x^3 \to -\infty$, and the local minimum at $(1,1)$ is not global — exactness of the quadratic expansion is what failed.

**6.** Expanding the squared norm:

$$
f(u,v) = u^2\norm{C_1}^2 + 2uv\,\langle C_1, C_2\rangle +
v^2\norm{C_2}^2 - 2u\,\langle C_1, b\rangle - 2v\,\langle C_2,
b\rangle + \norm b^2 ,
$$

a quadratic function of $(u, v)$ whose quadratic part is $\frac12(ru^2 + 2suv + tv^2)$ with $r = 2\norm{C_1}^2$, $s =
2\langle C_1, C_2\rangle$, $t = 2\norm{C_2}^2$.

**7.** $rt - s^2 = 4\bigl(\norm{C_1}^2\norm{C_2}^2 -
\langle C_1, C_2\rangle^2\bigr) \geq 0$ by Cauchy–Schwarz ([Theorem 23.4](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-cs)), with equality exactly when $C_1, C_2$ are proportional (or one is zero), i.e. when the pair is linked. So $rt - s^2 > 0 \iff (C_1, C_2)$ [free](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#def-b1-vspaces-free).

**8.** $\nabla f = 0$ reads

$$
\norm{C_1}^2 u + \langle C_1, C_2\rangle v = \langle C_1,
b\rangle,
\qquad
\langle C_1, C_2\rangle u + \norm{C_2}^2 v = \langle C_2,
b\rangle,
$$

the [normal equations](#pb-b1-multivar-1) with the Gram matrix on the left; they say $\langle uC_1 + vC_2 - b,\ C_i\rangle = 0$ for $i = 1, 2$, i.e. $b - p \perp \operatorname{Vect}(C_1, C_2)$ for $p = uC_1 +
vC_2$. By Part I (questions 3, 5) the unique [critical point](#thm-b1-multivar-critical) is the unique global minimizer, and by [Theorem 23.10](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) the characterization “$p \in
F$, $b - p \perp F$” identifies $p$ as the [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) of $b$ onto $F = \operatorname{Vect}(C_1, C_2)$.

**9.** $\norm{b - p}^2 = \norm b^2 - 2\langle b, p\rangle +
\norm p^2$, and $\langle p, b - p\rangle = 0$ gives $\norm p^2 =
\langle p, b\rangle$: the minimum equals $\norm b^2 - \langle p,
b\rangle$. Pythagoras: $\norm b^2 = \norm p^2 + \norm{b - p}^2$ — the data vector splits into its explained part $p$ and its residual part $b - p$, [orthogonal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) to each other.

**10.** $E(\alpha, \beta) = \norm{\alpha C_1 + \beta C_2 -
b}^2$ with the stated $C_1, C_2, b$; the [normal equations](#pb-b1-multivar-1) of question 8 are, entrywise,

$$
n\alpha + \Bigl(\sum x_i\Bigr)\beta = \sum y_i,
\qquad
\Bigl(\sum x_i\Bigr)\alpha + \Bigl(\sum x_i^2\Bigr)\beta = \sum
x_i y_i .
$$

**11.** Dividing by $n$: $\alpha + \beta\bar x = \bar y$ and $\alpha\bar x + \beta\bigl(v_x + \bar x^2\bigr) = c_{xy} +
\bar x\bar y$. Substituting $\alpha = \bar y - \beta\bar x$ into the second: $\beta v_x = c_{xy}$, so

$$
\beta = \frac{c_{xy}}{v_x}, \qquad \alpha = \bar y - \beta\bar
x ,
$$

and the first equation says precisely that $(\bar x, \bar y)$ lies on the line.

**12.** $v_x = \frac1n\sum(x_i - \bar x)^2 \geq 0$, zero iff every $x_i$ equals $\bar x$. And $rt - s^2 = 4\bigl(n\sum
x_i^2 - (\sum x_i)^2\bigr) = 4n^2 v_x$: question 7’s freeness condition is $v_x > 0$, i.e. the $x_i$ not all equal.

**13.** With $\alpha = \bar y - \beta\bar x$, center the data ($\tilde x_i = x_i - \bar x$, $\tilde y_i = y_i - \bar y$):

$$
E(\alpha, \beta) = \sum_i(\tilde y_i - \beta\tilde x_i)^2
= n\,v_y - 2\beta\,n\,c_{xy} + \beta^2 n\,v_x ,
$$

minimized at $\beta = c_{xy}/v_x$ with value $E_{\min} =
n\bigl(v_y - \frac{c_{xy}^2}{v_x}\bigr) = n v_y(1 - \rho^2)$. Since $E_{\min} \geq 0$ and $v_y > 0$: $\rho^2 \leq 1$, and $\abs\rho = 1$ iff $E_{\min} = 0$, i.e. iff every residual vanishes: the points lie exactly on the line.

**14.** $n = 4$: $\bar x = \frac64 = 1.5$, $\bar y = 2$, $\sum x_i^2 = 14$ so $v_x = 3.5 - 2.25 = 1.25$, $\sum x_iy_i =
0 + 1 + 6 + 12 = 19$ so $c_{xy} = 4.75 - 3 = 1.75$. Hence

$$
\beta = \frac{1.75}{1.25} = 1.4,
\qquad
\alpha = 2 - 1.4\times1.5 = -0.1 :
\qquad y = 1.4\,x - 0.1 .
$$

Fitted values $-0.1,\ 1.3,\ 2.7,\ 4.1$; residuals $0.1,\ -0.3,\
0.3,\ -0.1$; $E_{\min} = 0.01 + 0.09 + 0.09 + 0.01 = 0.2$ (check: $v_y = \frac{26}4 - 4 = 2.5$ and $4(2.5 -
\frac{1.75^2}{1.25}) = 4(2.5 - 2.45) = 0.2$).

**15.** The two [normal equations](#pb-b1-multivar-1) of question 10 are exactly $\sum_i\varepsilon_i = 0$ and $\sum_i x_i\varepsilon_i =
0$: the residual vector is [orthogonal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) to $C_1 = (1, \dots, 1)$ and to $C_2 = (x_i)$ — to the whole model space. In particular the optimal line always balances its errors: they sum to zero.

**16.** $\frac{\dd}{\dd\alpha}\sum(y_i - \alpha)^2 =
-2\sum(y_i - \alpha) = 0$ gives $\alpha = \bar y$ (and the second [derivative](https://one-course.com/books/math/3/en/chapter/14-differentiation#def-b1-derivative-def) $2n > 0$ makes it the global minimum, the function being a coercive quadratic in one variable). Minimal value $\sum(y_i - \bar y)^2 = n v_y$: the variance is the irreducible quadratic error of a constant model.

**17.** $\frac{\dd}{\dd\beta}\sum(y_i - \beta x_i)^2 =
-2\sum x_i(y_i - \beta x_i) = 0$ gives $\beta_0 = \frac{\sum
x_iy_i}{\sum x_i^2}$. Writing both slopes over centered quantities: $\beta_0 = \frac{c_{xy} + \bar x\bar y}{v_x + \bar
x^2}$ equals $\frac{c_{xy}}{v_x}$ iff $c_{xy}\bar x^2 = \bar
x\bar y\,v_x$, i.e. iff $\bar x = 0$ (centered abscissas) or $\bar y = \beta\bar x$ — the latter meaning $\alpha = 0$: the full [regression line](#pb-b1-multivar-1) already passes through the origin.

**18.** $g(\tau) = \norm{M - P_0}^2 - 2\tau\langle M - P_0,
w\rangle + \tau^2$ is minimal at $\tau^* = \langle M - P_0,
w\rangle$, with value $d(M,D)^2 = \norm{M - P_0}^2 - \langle M -
P_0, w\rangle^2$. In the plane, complete $w$ into an [orthonormal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) [basis](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#def-b1-vspaces-free) $(w, n)$ with $n = \frac{(a, b)}{\sqrt{a^2+b^2}}$ (unit normal of $D$): then $\norm{M - P_0}^2 = \langle M - P_0,
w\rangle^2 + \langle M - P_0, n\rangle^2$, so $d(M, D) =
\abs{\langle M - P_0, n\rangle}$, and with $aP_{0x} + bP_{0y} =
-c$:

$$
d = \frac{\abs{a x_0 + b y_0 + c}}{\sqrt{a^2 + b^2}} .
$$

**19.** From $E_{\min}/n = v_y - \frac{c_{xy}^2}{v_x}$ and $\beta^2 v_x = \frac{c_{xy}^2}{v_x}$:

$$
v_y = \beta^2 v_x + \frac{E_{\min}}n :
$$

total variance $=$ variance along the fitted line $+$ residual variance. (Dividing by $v_y$: $1 = \rho^2 + (1 - \rho^2)$, the share of variance “explained” by the line is $\rho^2$.)

**20.** The model space is $\operatorname{Vect}\bigl(C_1, C_2, C_3\bigr)$ with $C_1 = (1)$, $C_2 = (x_i)$, $C_3 = (x_i^2)$; minimizing $\norm{aC_1 + bC_2 +
cC_3 - b}^2$ leads, exactly as in question 8, to the $3\times3$ Gram system, whose matrix has entries $\langle C_i, C_j\rangle =
\sum_k x_k^{\,i+j-2}$: the displayed moment matrix. It is invertible iff $(C_1, C_2, C_3)$ is [free](https://one-course.com/books/math/3/en/chapter/18-vector-spaces#def-b1-vspaces-free) ([Exercise 23.11](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#exo-b1-euclid-11)); a relation $a + bx_k + cx_k^2 = 0$ for all $k$ makes every $x_k$ a root of one [polynomial](https://one-course.com/books/math/3/en/chapter/8-polynomials#def-b1-poly-def) of degree $\leq 2$, impossible with three distinct values unless $a
= b = c = 0$. Compare the moment matrices of the weekend problem of [Chapter 22](https://one-course.com/books/math/3/en/chapter/22-determinants-and-linear-systems#ch-b1-det), where their determinants were squared Vandermonde values.

**21.** First: $r = 2, s = 1, t = 2$, $rt - s^2 = 3 > 0$, $r > 0$: strict global minimum at the origin. Second: $r = 2, s
= 3, t = 2$, $rt - s^2 = -5 < 0$: saddle. Third: $r = s = t =
2$, $rt - s^2 = 0$: degenerate — but $x^2 + 2xy + y^2 = (x +
y)^2 \geq 0$ vanishes on the whole line $y = -x$: a (non-strict) global minimum attained along a line, invisible to the determinant test.

**22.** New sums ($n = 5$): $\bar x = 3.2$, $\bar y =
1.6$, $\sum x_iy_i = 19$ so $c_{xy} = 3.8 - 5.12 = -1.32$, $\sum x_i^2 = 114$ so $v_x = 22.8 - 10.24 = 12.56$. New slope:

$$
\beta = \frac{-1.32}{12.56} \approx -0.105 :
$$

one point turned a clearly increasing trend ($\beta = 1.4$) into a slightly decreasing one. The squared error charges a residual $\varepsilon^2$, so a single distant point — large $\abs{x_{i} - \bar x}$ *and* large residual — dominates both $c_{xy}$ and $v_x$: [least squares](#pb-b1-multivar-1) is efficient but not robust.

**23.** Set $W = \sum w_i$ and define $\bar x_w =
\frac1W\sum w_ix_i$, $\bar y_w$ likewise, $v_x^w = \frac1W\sum
w_ix_i^2 - \bar x_w^2$, $c^w_{xy} = \frac1W\sum w_ix_iy_i - \bar
x_w\bar y_w$. The [map](https://one-course.com/books/math/3/en/chapter/1-logic-sets-and-maps#def-b1-logic-map) $\langle u, v\rangle_w = \sum_i w_iu_iv_i$ is an [inner product](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-def) on $\R^n$ ($w_i > 0$ gives definiteness), so Part II applies verbatim and the [normal equations](#pb-b1-multivar-1) divide by $W$ into

$$
\alpha + \beta\bar x_w = \bar y_w,
\qquad
\alpha\bar x_w + \beta(v^w_x + \bar x_w^2) = c^w_{xy} + \bar
x_w\bar y_w ,
$$

whence $\beta = c^w_{xy}/v^w_x$ and $\alpha = \bar y_w -
\beta\bar x_w$: the same formulas, with every average weighted.

**24.** $E_{\min} = \sum\varepsilon_i^2 = 0$ forces every $\varepsilon_i = 0$: all points exactly on the line; by question 13 this is the case $\abs\rho = 1$. Test: for $(1,1), (2,3),
(3,5)$: $\bar x = 2$, $\bar y = 3$, $v_x = \frac{14}3 - 4 =
\frac23$, $c_{xy} = \frac{22}3 - 6 = \frac43$: $\beta = 2$, $\alpha = -1$, and indeed $y_i = 2x_i - 1$ for all three points; $v_y = \frac{35}3 - 9 = \frac83$ and $\rho^2 =
\frac{(4/3)^2}{(2/3)(8/3)} = 1$.

**25.** (i) For quadratic functions the order-two expansion is an *identity*, so the sign study of the quadratic form — pure algebra, questions 1–2 — classifies [critical points](#thm-b1-multivar-critical) with no admitted Taylor theorem. (ii) The [normal equations](#pb-b1-multivar-1) say “residual [orthogonal](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#def-b1-euclid-orthogonal) to the model space”, so the analytic minimum *is* the [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) of the data vector: calculus and Euclidean geometry compute the same object. (iii) The correlation $\rho = c_{xy}/\sqrt{v_xv_y}$ measures the share $\rho^2$ of the variance explained by the line, and Cauchy–Schwarz bounds it: $\abs\rho \leq 1$, with equality only for aligned data. (iv) Bases and freeness (Chapter 18), Gram and moment matrices and their determinants (Chapters 22–23), and [orthogonal projection](https://one-course.com/books/math/3/en/chapter/23-euclidean-spaces#thm-b1-euclid-projection) (Chapter 23) all reappeared as the working parts of one algorithm. Parts II–III develop the *method of [least squares](#pb-b1-multivar-1)* (Legendre, Gauss) through its *[normal equations](#pb-b1-multivar-1)*.