University Mathematics — Year 3 · Bachelor Year 3

23Characteristic Functions and the Central Limit Theorem

The law of large numbers says averages converge; the central limit theorem says how they fluctuate: the error, magnified by $\sqrt n$ , is asymptotically Gaussian — whatever the law one started from. This universality is the deepest fact of elementary probability, and its natural proof is Fourier-analytic: the characteristic function (the Fourier transform of a law) converts independent sums into products, and Chapter 14’s machinery — injectivity, Gaussian fixed points — converts pointwise convergence of these products into convergence of laws (Lévy’s theorem, proved in full). The chapter ends with Gaussian vectors and the honest derivation of the confidence intervals used everywhere in statistics; the weekend problem gives Lindeberg’s second proof of the CLT, with an explicit error rate.

23.1 Characteristic functions

Definition 23.1

The characteristic function of a real random variable $X$ is

\varphi_X(\xi) = \E\bigl[\eu^{\iu\xi X}\bigr] = \int_\R \eu^{\iu\xi x}\,\dd\P_X(x) \qquad (\xi \in \R)

(the transfer theorem computes it from the law; for a density $f$ , $\varphi_X(\xi) = \hat f(-\xi)$ in Chapter 14’s convention).

Proposition 23.2

(a) $\varphi_X(0) = 1$ , $\abs{\varphi_X} \leq 1$ , and $\varphi_X$ is uniformly continuous; $\varphi_{aX + b}(\xi) = \eu^{\iu b\xi}\varphi_X(a\xi)$ . (b) If $X, Y$ are independent: $\varphi_{X+Y} = \varphi_X\,\varphi_Y$ . (c) If $\E\abs X^k < \infty$ , then $\varphi_X \in \mathcal C^k$ with $\varphi_X^{(j)}(0) = \iu^j\,\E[X^j]$ for $j \leq k$ ; in particular, for centered $X \in L^2$ with variance $\sigma^2$ :

\varphi_X(\xi) = 1 - \frac{\sigma^2\xi^2}{2} + o(\xi^2) \qquad (\xi \to 0).

(d) Gaussian: $X \sim \mathcal N(m, \sigma^2)$ has $\varphi_X(\xi) = \eu^{\iu m\xi - \sigma^2\xi^2/2}$ .

Proof. (a) Bounds are immediate; continuity: $\abs{\varphi(\xi + h) - \varphi(\xi)} \leq \E\abs{\eu^{\iu hX} - 1} \to 0$ as $h \to 0$ by dominated convergence, uniformly in $\xi$ . The affine rule is a substitution. (b) $\eu^{\iu\xi(X+Y)} = \eu^{\iu\xi X}\eu^{\iu\xi Y}$ , and expectations of products of independent variables factor (Theorem 22.5, applied to real and imaginary parts). (c) Differentiation under the expectation, dominated by $\E\abs X^j$ (Theorem 10.15); the Taylor expansion at $0$ is then Taylor–Young for the $\mathcal C^2$ function $\varphi$ . (d) For $\mathcal N(0,1)$ : the Gaussian transform (Example 14.2 with $a = \frac12$ ) gives $\int\eu^{\iu\xi x}\frac{\eu^{-x^2/2}}{\sqrt{2\pi}}\dd x = \eu^{-\xi^2/2}$ ; the general case by the affine rule. ∎

Theorem 23.3 (Injectivity)

If $\varphi_X = \varphi_Y$ , then $X$ and $Y$ have the same law. More precisely, for $N \sim \mathcal N(0,1)$ independent of $X$ and $\varepsilon > 0$ , the smoothed variable $X + \varepsilon N$ has the density

p_\varepsilon(x) = \frac1{2\pi}\int_\R \varphi_X(-\xi)\,\eu^{-\varepsilon^2\xi^2/2}\, \eu^{\iu\xi x}\,\dd\xi ,

determined by $\varphi_X$ alone; letting $\varepsilon \to 0$ recovers the law of $X$ .

Proof. $X + \varepsilon N$ has the density $p_\varepsilon(x) = \E\bigl[g_\varepsilon(x - X)\bigr]$ , where $g_\varepsilon$ is the $\mathcal N(0, \varepsilon^2)$ density: indeed for Borel $B$ , independence and Tonelli give $\P(X + \varepsilon N \in B) = \int\!\!\int\mathbf 1_B(x + \varepsilon n)g_1(n)\,\dd n\,\dd\P_X(x) = \int_B\E[g_\varepsilon(t - X)]\dd t$ (substitute, then Tonelli again). Writing $g_\varepsilon$ by Fourier inversion of its transform (Exercise 14.4, rescaled): $g_\varepsilon(u) = \frac1{2\pi}\int \eu^{-\varepsilon^2\xi^2/2}\eu^{\iu\xi u}\dd\xi$ , and Fubini (everything dominated by the Gaussian factor):

p_\varepsilon(x) = \frac1{2\pi}\int_\R \varphi_X(-\xi)\,\eu^{-\varepsilon^2\xi^2/2}\, \eu^{\iu\xi x}\,\dd\xi ,

a functional of $\varphi_X$ alone. If $\varphi_X = \varphi_Y$ : $X + \varepsilon N$ and $Y + \varepsilon N$ have equal laws for every $\varepsilon$ ; for bounded continuous $f$ , $\E f(X + \varepsilon N) \to \E f(X)$ as $\varepsilon \to 0$ (dominated convergence, $X + \varepsilon N \to X$ pointwise on the product space), so $\E f(X) = \E f(Y)$ for all such $f$ — and this determines the law: for each $t$ , squeeze $\mathbf 1_{\intoc{-\infty}t}$ between the bounded continuous ramps $f_k^\pm$ (equal to $1$ on $\intoc{-\infty}{t \mp \frac1k}$ , to $0$ beyond $t \pm \frac1k$ , affine between); passing to the limit in $\E f_k^-(X) \leq F_X(t) \leq \E f_k^+(X)$ gives $F_X(t) = F_Y(t)$ at every $t$ where both are continuous, hence everywhere by right-continuity and density of common continuity points (both $F$ ’s have countably many jumps); equal distribution functions force equal laws (Exercise 9.3, resting on Theorem 9.7). ∎

23.2 Convergence in distribution

Definition 23.4

$X_n$ converges in distribution (or in law) to $X$ , written $X_n \Rightarrow X$ , if

\E\bigl[f(X_n)\bigr] \longrightarrow \E\bigl[f(X)\bigr] \qquad\text{for every bounded continuous } f\colon\R\to\R .

Equivalently (Exercise 23.4): $F_{X_n}(t) \to F_X(t)$ at every continuity point $t$ of $F_X$ . The $X_n$ need not live on a common probability space: only laws matter.

Theorem 23.5 (Helly’s selection theorem)

Every sequence $(F_n)$ of distribution functions has a subsequence converging pointwise, at every continuity point of the limit, to a nondecreasing right-continuous $G \colon \R \to \intcc01$ — possibly with $G(+\infty) - G(-\infty) < 1$ (mass may escape to infinity).

Proof. Diagonal extraction gives $F_{n_k}(q) \to \ell(q)$ for every rational $q$ (values in the compact $\intcc01$ ). Define $G(t) = \inf\{\ell(q) : q \in \Q, q > t\}$ : nondecreasing; right-continuous (an infimum over shrinking rational neighborhoods from the right). At a continuity point $t$ of $G$ : for rationals $q_1 < t < q_2$ ,

\ell(q_1) \leq \liminf F_{n_k}(t) \leq \limsup F_{n_k}(t) \leq \ell(q_2),

by monotonicity of each $F_{n_k}$ . From the definition of $G$ as an infimum and monotonicity of $\ell$ on the rationals: $G(s) \leq \ell(q) \leq G(q)$ whenever $s < q$ . Taking $s < q_1 < t$ gives $\ell(q_1) \geq G(s)$ , and $\ell(q_2) \leq G(q_2)$ ; letting $s \uparrow t$ and $q_2 \downarrow t$ , continuity of $G$ at $t$ squeezes both the $\liminf$ and the $\limsup$ to $G(t)$ . ∎

Lemma 23.6 (Tightness from the characteristic function)

For any random variable $X$ and $u > 0$ :

\P\Bigl(\abs X \geq \frac2u\Bigr) \;\leq\; \frac1u\int_{-u}^{u}\bigl(1 - \operatorname{Re}\varphi_X(\xi)\bigr)\,\dd\xi .

Proof. By Tonelli–Fubini (integrand bounded, region finite in $\xi$ ):

\frac1u\int_{-u}^u\bigl(1 - \operatorname{Re}\varphi_X(\xi)\bigr)\dd\xi = \E\Bigl[\frac1u\int_{-u}^u(1 - \cos(\xi X))\,\dd\xi\Bigr] = 2\,\E\Bigl[1 - \frac{\sin(uX)}{uX}\Bigr]

(interpret the bracket as its limit $0$ at $X = 0$ ). The integrand is nonnegative ( $\abs{\sin t} \leq \abs t$ ), and for $\abs{uX} \geq 2$ : $1 - \frac{\sin(uX)}{uX} \geq 1 - \frac1{\abs{uX}} \geq \frac12$ . Keeping only the event $\{\abs{uX} \geq 2\}$ inside the expectation therefore leaves at least $2 \cdot \frac12\,\P(\abs X \geq \frac2u)$ , which is the claim. ∎

Theorem 23.7 (Lévy’s continuity theorem)

Let $(X_n)$ be random variables whose characteristic functions converge pointwise: $\varphi_{X_n}(\xi) \to \varphi(\xi)$ for every $\xi$ , where $\varphi = \varphi_X$ is the characteristic function of some random variable $X$ . Then $X_n \Rightarrow X$ .

Proof. Tightness. Fix $\varepsilon > 0$ . Since $\varphi$ is continuous at $0$ with $\varphi(0) = 1$ , choose $u > 0$ with $\frac1u\int_{-u}^u(1 - \operatorname{Re}\varphi) < \varepsilon$ ; by dominated convergence (integrand bounded by $2$ on the fixed $[-u,u]$ ), the same integral for $\varphi_{X_n}$ is $< 2\varepsilon$ for $n$ large: Lemma 23.6 gives $\P(\abs{X_n} \geq \frac2u) \leq 2\varepsilon$ for large $n$ , and enlarging the constant handles the finitely many others: the laws are tight — no mass escapes.

Subsequences. Let $(F_{n_k})$ be any subsequence; by Helly (Theorem 23.5) extract $F_{n_{k_j}} \to G$ at continuity points. Tightness forces $G(-\infty) = 0$ , $G(+\infty) = 1$ ( $G(\frac2u) - G(-\frac2u) \geq 1 - 2\varepsilon$ at continuity points): $G$ is a genuine distribution function, of some random variable $Y$ . Then $X_{n_{k_j}} \Rightarrow Y$ (Exercise 23.4, distributional convergence from $F$ ’s), so $\varphi_{X_{n_{k_j}}} \to \varphi_Y$ pointwise ( $x \mapsto \eu^{\iu\xi x}$ is bounded continuous, real and imaginary parts separately); comparing with the hypothesis: $\varphi_Y = \varphi = \varphi_X$ , and injectivity (Theorem 23.3) gives $Y \sim X$ , i.e. $G = F_X$ .

Conclusion. Every subsequence of $(F_n)$ has a sub-subsequence converging to the same $F_X$ (at its continuity points); hence $F_n(t) \to F_X(t)$ at every continuity point $t$ (a real sequence all of whose subsequences have subsubsequences with the same limit converges): $X_n \Rightarrow X$ . ∎

23.3 The central limit theorem

Theorem 23.8 (Central limit theorem)

Let $(X_n)$ be i.i.d. with $\E X_1 = m$ and $\V(X_1) = \sigma^2 \in \intoo0\infty$ . Then

\frac{S_n - nm}{\sigma\sqrt n} \;\Longrightarrow\; \mathcal N(0, 1) : \qquad \P\Bigl(a \leq \frac{S_n - nm}{\sigma\sqrt n} \leq b\Bigr) \longrightarrow \frac{1}{\sqrt{2\pi}}\int_a^b\eu^{-x^2/2}\,\dd x

for all $a < b$ .

Proof. Center and normalize: $Z_i = \frac{X_i - m}{\sigma}$ (i.i.d., mean $0$ , variance $1$ ) and $T_n = \frac1{\sqrt n}\sum_{i\leq n}Z_i$ . By independence and the affine rule (Proposition 23.2):

\varphi_{T_n}(\xi) = \varphi_{Z}\Bigl(\frac{\xi}{\sqrt n}\Bigr)^{n}, \qquad \varphi_Z(\eta) = 1 - \frac{\eta^2}2 + \eta^2\rho(\eta),\quad \rho(\eta)\to0 .

Fix $\xi$ and let $a_n = \varphi_Z(\xi/\sqrt n)$ , $b_n = 1 - \frac{\xi^2}{2n}$ : both have modulus $\leq 1$ for $n$ large ( $\abs{b_n} \leq 1$ once $\xi^2 \leq 4n$ ; $\abs{a_n} \leq 1$ always). The elementary inequality $\abs{a^n - b^n} \leq n\abs{a - b}$ for $\abs a, \abs b \leq 1$ (telescoping $a^n - b^n = \sum a^k(a - b)b^{n-1-k}$ ) gives

\Bigl|\varphi_{T_n}(\xi) - \Bigl(1 - \frac{\xi^2}{2n}\Bigr)^{n}\Bigr| \leq n\,\Bigl|\varphi_Z\Bigl(\frac\xi{\sqrt n}\Bigr) - 1 + \frac{\xi^2}{2n}\Bigr| = \xi^2\,\Bigl|\rho\Bigl(\frac{\xi}{\sqrt n}\Bigr)\Bigr| \longrightarrow 0,

while $\bigl(1 - \frac{\xi^2}{2n}\bigr)^n \to \eu^{-\xi^2/2}$ (real logarithm). So $\varphi_{T_n}(\xi) \to \eu^{-\xi^2/2} = \varphi_{\mathcal N(0,1)}(\xi)$ (Proposition 23.2(d)) for every $\xi$ : Lévy (Theorem 23.7) concludes $T_n \Rightarrow \mathcal N(0,1)$ . The interval probabilities follow since $F_{\mathcal N}$ is continuous everywhere. ∎

Example 23.9 (Confidence intervals, honestly derived)

Poll $n$ independent voters; $\hat p_n = S_n/n$ estimates the true $p$ , with $\sigma^2 = p(1-p) \leq \frac14$ . The CLT gives, for large $n$ ,

\P\Bigl(\abs{\hat p_n - p} \leq \frac{z}{2\sqrt n}\Bigr) \;\geq\; \P\Bigl(\Bigl|\frac{S_n - np}{\sigma\sqrt n}\Bigr| \leq z\Bigr) \longrightarrow \Phi(z) - \Phi(-z),

where $\Phi$ is the standard Gaussian distribution function. With $z = 1.96$ : asymptotic confidence $95\%$ , and margin $\frac{1.96}{2\sqrt n} \leq 3\%$ requires $n \geq \bigl(\frac{1.96}{0.06}\bigr)^2 \approx 1068$ — the number behind every “ $\pm3$ points, $95\%$ ” one reads; compare Chebyshev’s $5556$ (Exercise 22.7). The $\sqrt n$ is universal: to halve the error, quadruple the sample — the same law that fixes Monte Carlo’s cost (Exercise 23.7).

23.4 Gaussian vectors

Definition 23.10

A random vector $X = (X_1, \dots, X_d)$ is Gaussian if every linear combination $\langle t, X\rangle = \sum t_iX_i$ is a (possibly degenerate) real Gaussian variable. Its law is determined by the mean vector $m = (\E X_i)$ and the covariance matrix $\Sigma = \bigl(\operatorname{Cov} (X_i, X_j)\bigr)$ : indeed the characteristic function of the vector, $\varphi_X(t) = \E\eu^{\iu\langle t, X\rangle}$ , is the value at $1$ of the cf of $\langle t, X\rangle$ :

\varphi_X(t) = \exp\Bigl(\iu\langle t, m\rangle - \tfrac12\,t^{\mathsf T}\Sigma\,t\Bigr),

and $d$ -dimensional characteristic functions are injective (same smoothing proof as Theorem 23.3, coordinatewise Gaussians).

Theorem 23.11

Let $X$ be a Gaussian vector.

Every affine image $AX + b$ is a Gaussian vector.
The components $X_i$ are independent if and only if $\Sigma$ is diagonal: for jointly Gaussian variables, uncorrelated $=$ independent.
If $\Sigma$ is invertible, $X$ has the density $\frac{1}{(2\pi)^{d/2}\sqrt{\det\Sigma}} \exp\bigl(-\frac12(x - m)^{\mathsf T}\Sigma^{-1}(x - m)\bigr)$ .

Proof. (1) Linear combinations of components of $AX + b$ are affine functions of linear combinations of $X$ : Gaussian (an affine image of a Gaussian variable is Gaussian). (2) If $\Sigma$ is diagonal, the characteristic function factorizes: $\varphi_X(t) = \prod_i\exp(\iu t_im_i - \frac12\Sigma_{ii}t_i^2) = \prod\varphi_{X_i}(t_i)$ , which is the characteristic function of the product law (Theorem 22.5 read through $d$ -dimensional injectivity): the components are independent. The converse is the vanishing of covariances of independent $L^2$ variables. (3) Diagonalize $\Sigma = P D P^{\mathsf T}$ ( $P$ orthogonal, $D > 0$ diagonal — Exercise 20.8); the vector $Y = P^{\mathsf T}(X - m)$ is Gaussian with covariance $D$ : by (2) its components are independent $\mathcal N(0, d_i)$ , so $Y$ has the product density; push forward by the volume-preserving $x = m + PY$ (Theorem 11.10, $\abs{\det P} = 1$ ) and rewrite the exponent invariantly. ∎

Theorem 23.12 (Multidimensional CLT)

Let $(X_n)$ be i.i.d. square-integrable random vectors of $\R^d$ with mean $m$ and covariance matrix $\Sigma$ . Then $\frac{S_n - nm}{\sqrt n}$ converges in distribution to the Gaussian vector $\mathcal N(0, \Sigma)$ .

Proof. Admitted at this level. ∎

Remark 23.13

Almost everything is already in our hands. For each direction $t \in \R^d$ , the real variable $\langle t, \frac{S_n - nm}{\sqrt n}\rangle$ is a normalized sum of i.i.d. real variables of variance $t^{\mathsf T}\Sigma t$ , so Theorem 23.8’s computation gives pointwise convergence of the $d$ -dimensional characteristic functions to $\eu^{-t^{\mathsf T}\Sigma t/2}$ , the characteristic function of $\mathcal N(0, \Sigma)$ (Definition 23.10). What we have not re-proved is Lévy’s continuity theorem in $\R^d$ : Helly’s selection and the tightness estimate generalize routinely (coordinatewise), and this Cramér–Wold reduction is carried out honestly in any graduate probability course; nothing beyond this chapter’s methods is needed.

Method 23.14

To identify a limit law: compute characteristic functions, take the pointwise limit, recognize it (Gaussian $\eu^{-\sigma^2\xi^2/2}$ , Poisson $\eu^{\lambda(\eu^{\iu\xi}-1)}$ , exponential $\frac{\lambda} {\lambda - \iu\xi}$ , …) and invoke Lévy. The three-step ritual (independence $\to$ product; Taylor at $0$ $\to$ exponential limit; Lévy $\to$ convergence in law) proves the CLT, the Poisson law of rare events (Exercise 23.5), and every classical limit theorem of this course. For a.s. statements, go back to Chapter 22’s toolkit: the two chapters answer different questions about the same $S_n$ .

23.5 Exercises

Exercise 23.1 ★

Compute the characteristic functions: uniform on $\intcc{-1}1$ ; exponential $\mathcal E(\lambda)$ ; Poisson $\mathcal P(\lambda)$ ; binomial $\mathcal B(n, p)$ . Deduce via Theorem 23.3 that the sum of independent Poisson variables ( $\lambda, \mu$ ) is Poisson $(\lambda + \mu)$ .

Solution

Solution of Exercise 23.1.

Uniform on $\intcc{-1}1$ : $\varphi(\xi) = \frac12\int_{-1}^1\eu^{\iu\xi x}\dd x = \frac{\sin\xi}{\xi}$ (equal to $1$ at $\xi = 0$ ). Exponential $\mathcal E(\lambda)$ : $\varphi(\xi) = \lambda\int_0^\infty\eu^{(\iu\xi - \lambda)x}\dd x = \frac{\lambda}{\lambda - \iu\xi}$ (the antiderivative vanishes at $+\infty$ since $\operatorname{Re}(\iu\xi - \lambda) < 0$ ). Poisson $\mathcal P(\lambda)$ : by the transfer theorem for discrete laws,

\varphi(\xi) = \sum_{k\geq0}\eu^{\iu\xi k}\,\eu^{-\lambda}\frac{\lambda^k}{k!} = \eu^{-\lambda}\exp\bigl(\lambda\eu^{\iu\xi}\bigr) = \exp\bigl(\lambda(\eu^{\iu\xi} - 1)\bigr).

Binomial $\mathcal B(n, p)$ : a sum of $n$ independent Bernoulli variables, each with cf $1 - p + p\eu^{\iu\xi}$ , so $\varphi(\xi) = \bigl(1 - p + p\eu^{\iu\xi}\bigr)^n$ (Proposition 23.2(b)). Poisson additivity: if $X \sim \mathcal P(\lambda)$ , $Y \sim \mathcal P(\mu)$ are independent,

\varphi_{X+Y}(\xi) = \eu^{\lambda(\eu^{\iu\xi}-1)} \eu^{\mu(\eu^{\iu\xi}-1)} = \eu^{(\lambda+\mu)(\eu^{\iu\xi}-1)},

the cf of $\mathcal P(\lambda + \mu)$ ; injectivity (Theorem 23.3) identifies the law.

Exercise 23.2 ★★

(a) Show that $\varphi_X$ is real-valued if and only if $X$ and $-X$ have the same law (a symmetric variable). (b) Suppose $\abs{\varphi_X(\xi_0)} = 1$ for some $\xi_0 \neq 0$ . Show that $X$ is almost surely supported on an arithmetic progression $a + \frac{2\pi}{\xi_0}\Z$ (write $\varphi_X(\xi_0) = \eu^{\iu\theta}$ and compute $\E[1 - \cos(\xi_0X - \theta)]$ ). Deduce that if $X$ has a density, then $\abs{\varphi_X(\xi)} < 1$ for all $\xi \neq 0$ .

Solution

Solution of Exercise 23.2.

(a) $\overline{\varphi_X(\xi)} = \E\eu^{-\iu\xi X} = \varphi_{-X}(\xi)$ . So $\varphi_X$ is real if and only if $\varphi_X = \varphi_{-X}$ , if and only if (injectivity, Theorem 23.3) $X$ and $-X$ have the same law. (b) Write $\varphi_X(\xi_0) = \eu^{\iu\theta}$ . Then

\E\bigl[1 - \cos(\xi_0X - \theta)\bigr] = 1 - \operatorname{Re}\bigl(\eu^{-\iu\theta} \varphi_X(\xi_0)\bigr) = 1 - 1 = 0 .

The integrand is nonnegative, so $\cos(\xi_0X - \theta) = 1$ almost surely (a nonnegative variable with zero expectation vanishes a.s.), i.e. $\xi_0X - \theta \in 2\pi\Z$ a.s.: $X$ takes its values in the arithmetic progression $\frac{\theta}{\xi_0} + \frac{2\pi}{\xi_0}\Z$ almost surely. If $X$ has a density, this countable set is Lebesgue-null, so it carries probability $0$ — contradiction; therefore $\abs{\varphi_X(\xi)} < 1$ for every $\xi \neq 0$ .

Exercise 23.3 ★★

Let $X \sim \mathcal N(m_1, \sigma_1^2)$ and $Y \sim \mathcal N(m_2, \sigma_2^2)$ be independent. Show $X + Y \sim \mathcal N(m_1 + m_2, \sigma_1^2 + \sigma_2^2)$ , and more generally that the Gaussian family is stable under independent sums and affine maps. Contrast: is the sum of two dependent Gaussians always Gaussian? (Exercise 23.9.)

Solution

Solution of Exercise 23.3.

By independence and Proposition 23.2:

\varphi_{X+Y}(\xi) = \eu^{\iu m_1\xi - \sigma_1^2\xi^2/2}\, \eu^{\iu m_2\xi - \sigma_2^2\xi^2/2} = \eu^{\iu(m_1+m_2)\xi - (\sigma_1^2+\sigma_2^2)\xi^2/2},

the cf of $\mathcal N(m_1 + m_2, \sigma_1^2 + \sigma_2^2)$ ; injectivity concludes. Stability under affine maps is the affine rule ( $aX + b \sim \mathcal N(am_1 + b, a^2\sigma_1^2)$ , allowing the degenerate case $a = 0$ ), and stability under independent sums follows by induction on the computation above. For dependent Gaussians the sum need not be Gaussian: in Exercise 23.9, $X$ and $Y = \varepsilon X$ are each standard Gaussian but $X + Y$ vanishes with probability $\frac12$ without being a.s. zero, so it is not Gaussian.

Exercise 23.4 ★★

(a) Prove the equivalence in Definition 23.4: if $\E f(X_n) \to \E f(X)$ for all bounded continuous $f$ , then $F_{X_n}(t) \to F_X(t)$ at continuity points (squeeze $\mathbf 1_{\intoc{-\infty}t}$ between two continuous staircase-ramps); and conversely (approximate a bounded continuous $f$ by sums of ramp functions, or condition on a fine grid of continuity points) — the converse may be treated for $f$ uniformly continuous first, then in general. (b) Show that $X_n \Rightarrow c$ (a constant) implies $X_n \to c$ in probability.

Solution

Solution of Exercise 23.4.

(a) Direct implication. Let $t$ be a continuity point of $F_X$ and $\delta > 0$ . Take the continuous ramps $f^-$ ( $= 1$ on $\intoc{-\infty}{t-\delta}$ , $0$ from $t$ on, affine between) and $f^+$ ( $= 1$ on $\intoc{-\infty}t$ , $0$ from $t + \delta$ on, affine between); then $f^- \leq \mathbf 1_{\intoc{-\infty}t} \leq f^+$ , so

\E f^-(X_n) \leq F_{X_n}(t) \leq \E f^+(X_n),

and the outer terms converge to $\E f^\pm(X)$ , themselves squeezed between $F_X(t - \delta)$ and $F_X(t + \delta)$ . Letting $n \to \infty$ then $\delta \to 0$ and using continuity of $F_X$ at $t$ : $F_{X_n}(t) \to F_X(t)$ .

Converse. Let $f$ be bounded continuous, $M = \sup\abs f$ , $\varepsilon > 0$ . The continuity points of $F_X$ are dense ( $F_X$ has at most countably many jumps), so choose continuity points $a < b$ with $F_X(a) < \varepsilon$ and $1 - F_X(b) < \varepsilon$ . On the compact $\intcc ab$ the function $f$ is uniformly continuous: choose continuity points $a = t_0 < t_1 < \dots < t_m = b$ of $F_X$ with oscillation of $f$ at most $\varepsilon$ on each $\intoc{t_{j-1}}{t_j}$ , and set $g = \sum_j f(t_j)\,\mathbf 1_{\intoc{t_{j-1}}{t_j}}$ . Then $\abs{f - g} \leq \varepsilon$ on $\intoc ab$ , $\abs g \leq M$ , and for $T = X_n$ or $X$ :

\bigl|\E f(T) - \E g(T)\bigr| \leq \varepsilon + 2M\bigl(F_T(a) + 1 - F_T(b)\bigr).

Moreover $\E g(X_n) = \sum_j f(t_j)\bigl(F_{X_n}(t_j) - F_{X_n}(t_{j-1})\bigr) \to \E g(X)$ (a finite sum of converging terms, all $t_j$ being continuity points), and $F_{X_n}(a) \to F_X(a) < \varepsilon$ , $1 - F_{X_n}(b) \to 1 - F_X(b) < \varepsilon$ . Assembling: $\limsup_n\abs{\E f(X_n) - \E f(X)} \leq 2\varepsilon + 8M\varepsilon$ ; let $\varepsilon \to 0$ .

(b) The distribution function of the constant $c$ is $\mathbf 1_{\intco c\infty}$ , continuous except at $c$ . For $\varepsilon > 0$ , the points $c - \varepsilon$ and $c + \frac\varepsilon2$ are continuity points, so

\P(\abs{X_n - c} > \varepsilon) \leq F_{X_n}(c - \varepsilon) + 1 - F_{X_n}\Bigl(c + \frac\varepsilon2\Bigr) \longrightarrow 0 + 1 - 1 = 0 .

Exercise 23.5 ★★

(Law of rare events) Let $X_n \sim \mathcal B(n, p_n)$ with $np_n \to \lambda > 0$ . Show, via characteristic functions and Theorem 23.7, that $X_n \Rightarrow \mathcal P(\lambda)$ . Numerical sanity check: compare $\P(X = 0)$ for $\mathcal B(100, 0.02)$ and $\mathcal P(2)$ .

Solution

Solution of Exercise 23.5.

Let $z_n = p_n(\eu^{\iu\xi} - 1)$ , so $\varphi_{X_n}(\xi) = (1 + z_n)^n$ (Exercise 23.1) and $\abs{z_n} \leq 2p_n \to 0$ (note $p_n = \frac{np_n}n \to 0$ ). Both $1 + z_n$ and $\eu^{z_n}$ have modulus at most $1$ : $\abs{1 + z_n} = \abs{(1 - p_n) + p_n\eu^{\iu\xi}} \leq 1$ by the triangle inequality, and $\abs{\eu^{z_n}} = \eu^{p_n(\cos\xi - 1)} \leq 1$ . The telescoping inequality $\abs{a^n - b^n} \leq n\abs{a - b}$ (proof of Theorem 23.8) and the power series bound $\abs{\eu^z - 1 - z} \leq \abs z^2\eu^{\abs z}$ give

\bigl|(1 + z_n)^n - \eu^{nz_n}\bigr| \leq n\bigl|1 + z_n - \eu^{z_n}\bigr| \leq n\,\abs{z_n}^2\,\eu^{\abs{z_n}} \leq 4\eu^2\,np_n^2 = 4\eu^2\,(np_n)\,p_n \longrightarrow 0 .

Since $nz_n = np_n(\eu^{\iu\xi} - 1) \to \lambda(\eu^{\iu\xi} - 1)$ , we conclude $\varphi_{X_n}(\xi) \to \exp\bigl(\lambda(\eu^{\iu\xi} - 1)\bigr)$ for every $\xi$ : the cf of $\mathcal P(\lambda)$ , and Lévy (Theorem 23.7) gives $X_n \Rightarrow \mathcal P(\lambda)$ . Numerically: $\P\bigl(\mathcal B(100, 0.02) = 0\bigr) = 0.98^{100} = \eu^{100\ln 0.98} \approx \eu^{-2.020} \approx 0.1326$ , while $\P\bigl(\mathcal P(2) = 0\bigr) = \eu^{-2} \approx 0.1353$ : two percent apart already at this coarse $n$ .

Exercise 23.6 ★★

(a) A fair die is rolled $n = 1000$ times; approximate the probability that the total exceeds $3600$ (mean $3500$ , variance per roll $\frac{35}{12}$ ). (b) For $S \sim \mathcal B(100, \frac12)$ , approximate $\P(45 \leq S \leq 55)$ by the CLT with the continuity correction ( $\pm\frac12$ ), and comment on the correction’s effect.

Solution

Solution of Exercise 23.6.

(a) One roll has mean $\frac72$ and variance $\frac{35}{12}$ , so $S$ has mean $3500$ , variance $\frac{35000}{12} \approx 2916.7$ and standard deviation $\approx 54.0$ . By the CLT,

\P(S > 3600) = \P\Bigl(\frac{S - 3500}{54.0} > 1.85\Bigr) \approx 1 - \Phi(1.85) \approx 0.032 :

about a $3\%$ chance. (b) $S \sim \mathcal B(100, \frac12)$ : mean $50$ , standard deviation $5$ . With the continuity correction,

\P(45 \leq S \leq 55) \approx \Phi\Bigl(\frac{55.5 - 50}{5}\Bigr) - \Phi\Bigl(\frac{44.5 - 50}{5}\Bigr) = 2\Phi(1.1) - 1 \approx 0.729,

against the exact value $0.7287$ ; without the correction, $2\Phi(1) - 1 \approx 0.683$ , off by almost five points. The correction matters because $S$ is a lattice variable: the atom $\P(S = k)$ is well approximated by the Gaussian mass of $\intcc{k - \frac12}{k + \frac12}$ , and clipping the interval at the integers $45$ and $55$ discards half an atom at each end.

Exercise 23.7 ★★

(Monte Carlo error) In the setting of Problem 22.1, question 11, with $g \in L^2(\intcc01^d)$ , let $\sigma^2 = \V(g(U_1))$ and $I = \int g$ . Show

\sqrt n\,\Bigl(\frac1n\sum_{k\leq n}g(U_k) - I\Bigr) \Longrightarrow \mathcal N(0, \sigma^2),

and deduce the asymptotic $95\%$ error bar $\pm 1.96\,\sigma/\sqrt n$ — independent of the dimension $d$ . Compare with the deterministic midpoint rule in dimension $d$ (error $\sim n^{-2/d}$ for $\mathcal C^2$ integrands): from which dimension on does random sampling win?

Solution

Solution of Exercise 23.7.

The variables $g(U_k)$ are i.i.d. (measurable images of i.i.d. variables), square-integrable, with mean $I$ (transfer theorem, Exercise 11.9) and variance $\sigma^2$ . If $\sigma > 0$ , Theorem 23.8 applied to them is exactly the stated convergence

\sqrt n\,\Bigl(\frac1n\sum_{k\leq n}g(U_k) - I\Bigr) = \frac{\sum_{k\leq n}\bigl(g(U_k) - I\bigr)}{\sqrt n} \Longrightarrow \mathcal N(0, \sigma^2)

(if $\sigma = 0$ , $g$ is a.s. constant and the left-hand side vanishes identically). Hence $\P\bigl(\abs{\frac1n\sum g(U_k) - I} \leq 1.96\,\sigma/\sqrt n\bigr) \to 0.95$ : the error bar $\pm 1.96\,\sigma/\sqrt n$ sees the dimension $d$ only through the constant $\sigma$ , never through the rate in $n$ . The midpoint rule with $n$ nodes in dimension $d$ has mesh $n^{-1/d}$ and error of order $n^{-2/d}$ for $\mathcal C^2$ integrands. Monte Carlo’s $n^{-1/2}$ decays faster than $n^{-2/d}$ exactly when $\frac12 > \frac2d$ , i.e. $d > 4$ : from dimension $5$ on, random sampling asymptotically beats the grid — the curse of dimensionality spares probabilistic methods, which is why Monte Carlo rules high-dimensional integration.

Exercise 23.8 ★★★

(Slutsky) Suppose $X_n \Rightarrow X$ and $Y_n \to c$ in probability ( $c$ constant). Show $X_n + Y_n \Rightarrow X + c$ and $Y_nX_n \Rightarrow cX$ . (Work with characteristic functions and the bound $\abs{\E\eu^{\iu\xi (X_n+Y_n)} - \eu^{\iu\xi c}\E\eu^{\iu\xi X_n}} \leq \E\abs{\eu^{\iu\xi(Y_n - c)} - 1}$ , split on $\abs{Y_n - c} \leq \delta$ .) Application: in Example 23.9, justify replacing the unknown $\sigma = \sqrt{p(1-p)}$ by $\sqrt{\hat p_n(1 - \hat p_n)}$ .

Solution

Solution of Exercise 23.8.

Sum. For fixed $\xi$ :

\bigl|\E\eu^{\iu\xi(X_n+Y_n)} - \eu^{\iu\xi c}\,\E\eu^{\iu\xi X_n}\bigr| = \bigl|\E\bigl[\eu^{\iu\xi X_n}\bigl(\eu^{\iu\xi Y_n} - \eu^{\iu\xi c}\bigr)\bigr]\bigr| \leq \E\bigl|\eu^{\iu\xi(Y_n - c)} - 1\bigr| .

Split on the event $\{\abs{Y_n - c} \leq \delta\}$ : there, $\abs{\eu^{\iu\xi(Y_n-c)} - 1} \leq \abs\xi\,\delta$ (the chord is shorter than the arc); the complement contributes at most $2\,\P(\abs{Y_n - c} > \delta) \to 0$ . Hence the $\limsup$ is $\leq \abs\xi\,\delta$ for every $\delta > 0$ : the difference tends to $0$ . Since $\E\eu^{\iu\xi X_n} \to \varphi_X(\xi)$ , we get $\varphi_{X_n+Y_n}(\xi) \to \eu^{\iu\xi c}\varphi_X(\xi) = \varphi_{X+c}(\xi)$ , and Lévy (Theorem 23.7) yields $X_n + Y_n \Rightarrow X + c$ .

Product. First, $cX_n \Rightarrow cX$ : $\varphi_{cX_n}(\xi) = \varphi_{X_n}(c\xi) \to \varphi_X(c\xi) = \varphi_{cX}(\xi)$ . Next, $(Y_n - c)X_n \to 0$ in probability: the laws of the $X_n$ are tight (their cfs converge to a cf; see the tightness step of Theorem 23.7), so given $\varepsilon > 0$ pick $M$ with $\P(\abs{X_n} > M) \leq \varepsilon$ for all $n$ ; then

\P\bigl(\abs{(Y_n - c)X_n} > \varepsilon\bigr) \leq \P(\abs{X_n} > M) + \P\Bigl(\abs{Y_n - c} > \frac{\varepsilon}{M}\Bigr) \leq \varepsilon + o(1) .

Writing $Y_nX_n = cX_n + (Y_n - c)X_n$ and applying the sum part (whose proof only used $Y_n' := (Y_n - c)X_n \to 0$ in probability, with constant $0$ ): $Y_nX_n \Rightarrow cX$ .

Application. By the strong law of large numbers (Theorem 22.13), $\hat p_n \to p$ a.s., so by continuity $\hat\sigma_n = \sqrt{\hat p_n(1 - \hat p_n)} \to \sigma = \sqrt{p(1 - p)} > 0$ a.s., hence $\frac{\sigma}{\hat\sigma_n} \to 1$ in probability. Slutsky’s product rule upgrades $\frac{S_n - np}{\sigma\sqrt n} \Rightarrow \mathcal N(0,1)$ to $\frac{S_n - np}{\hat\sigma_n\sqrt n} = \frac{\sigma}{\hat\sigma_n}\cdot \frac{S_n - np}{\sigma\sqrt n} \Rightarrow \mathcal N(0,1)$ : the usable confidence interval $\hat p_n \pm 1.96\,\hat\sigma_n/\sqrt n$ , built from the data alone, keeps its asymptotic $95\%$ level.

Exercise 23.9 ★★★

Let $X \sim \mathcal N(0,1)$ and $\varepsilon$ independent with $\P(\varepsilon = \pm1) = \frac12$ ; set $Y = \varepsilon X$ . (a) Show $Y \sim \mathcal N(0,1)$ and $\operatorname{Cov}(X, Y) = 0$ . (b) Show $X$ and $Y$ are not independent, and that $(X, Y)$ is not a Gaussian vector (compute $\P(X + Y = 0)$ ). (c) Moral: Theorem 23.11(2) requires joint Gaussianity — “uncorrelated Gaussians” alone proves nothing.

Solution

Solution of Exercise 23.9.

(a) Splitting the expectation over the two values of $\varepsilon$ (independence): for Borel $B$ , $\P(Y \in B) = \frac12\P(X \in B) + \frac12\P(-X \in B) = \P(X \in B)$ , since $-X \sim X$ ( $\mathcal N(0,1)$ is symmetric): $Y \sim \mathcal N(0,1)$ . And $\operatorname{Cov}(X, Y) = \E[\varepsilon X^2] = \E[\varepsilon]\,\E[X^2] = 0 \cdot 1 = 0$ . (b) $\abs Y = \abs X$ , so $\P(\abs X \leq 1,\ \abs Y \geq 2) = 0$ while $\P(\abs X \leq 1)\,\P(\abs Y \geq 2) > 0$ : not independent. If $(X, Y)$ were a Gaussian vector, $X + Y = (1 + \varepsilon)X$ would be a real Gaussian variable (Definition 23.10 with $t = (1,1)$ ); but $\P(X + Y = 0) = \P(\varepsilon = -1) = \frac12$ , whereas a Gaussian variable has an atom only if it is a.s. constant — and $X + Y$ equals $2X \neq 0$ a.s. on $\{\varepsilon = 1\}$ . Contradiction: $(X, Y)$ is not Gaussian. (c) Each marginal is Gaussian and the covariance vanishes, yet independence fails — because the pair is not jointly Gaussian. Theorem 23.11(2) cannot be weakened to “Gaussian marginals”.

Exercise 23.10 ★★

The Cauchy law has density $\frac1{\pi(1 + x^2)}$ . (a) Show its characteristic function is $\eu^{-\abs\xi}$ (Exercise 14.1 and inversion). (b) Show that if $X_1, \dots, X_n$ are i.i.d. Cauchy, then $\frac{S_n}n$ is again Cauchy — the same law: the average never concentrates. (c) Reconcile with the laws of large numbers and the CLT: which hypotheses fail? (Compute $\E\abs{X_1}$ .)

Solution

Solution of Exercise 23.10.

(a) Exercise 14.1 computes $\widehat{\eu^{-\abs\cdot}}(\xi) = \frac{2}{1 + \xi^2}$ ; both sides being integrable, Fourier inversion (Theorem 14.5) turns this around:

\int_\R\eu^{\iu\xi x}\,\frac{\dd x}{\pi(1 + x^2)} = \eu^{-\abs\xi},

which is exactly $\varphi_X(\xi)$ for a Cauchy variable $X$ . (b) By independence, $\varphi_{S_n}(\xi) = \bigl(\eu^{-\abs\xi}\bigr)^n = \eu^{-n\abs\xi}$ , so $\varphi_{S_n/n}(\xi) = \varphi_{S_n}(\xi/n) = \eu^{-\abs\xi}$ : the empirical mean $\frac{S_n}n$ is again standard Cauchy for every $n$ (injectivity). The average never concentrates: its fluctuations at time $10^6$ are those of a single observation. (c) $\E\abs{X_1} = \frac2\pi\int_0^\infty\frac{x\,\dd x}{1 + x^2} = +\infty$ : the Cauchy law is not integrable, so the strong law of large numbers (Theorem 22.13) does not apply, and the CLT (which needs a finite variance) even less. Here their conclusions genuinely fail, not merely their proofs. Consistency check: $\varphi(\xi) = \eu^{-\abs\xi}$ is not differentiable at $0$ , as Proposition 23.2(c) read contrapositively predicts for a non-integrable variable.

Exercise 23.11 ★★

(Stable laws in embryo) Let $(X_n)$ be i.i.d. standard Cauchy (Exercise 23.10). (a) Show that for any $a, b > 0$ , $aX_1 + bX_2$ has the law of $(a + b)X_1$ : the Cauchy family is strictly stable of index $1$ . (b) Show that the Gaussian family is strictly stable of index $2$ : $aX_1 + bX_2 \sim \sqrt{a^2 + b^2}\,X_1$ for $X_i$ i.i.d. $\mathcal N(0,1)$ . (c) Explain, via characteristic functions of the form $\eu^{-c\abs\xi^\alpha}$ , why index- $\alpha$ stability forces the normalization $n^{1/\alpha}$ for sums, and what this says about the basins of attraction of the CLT: which i.i.d. sums can converge, after affine normalization, to a Cauchy law rather than a Gaussian?

Solution

Solution of Exercise 23.11.

(a) $\varphi_{aX_1 + bX_2}(\xi) = \eu^{-a\abs\xi}\eu^{-b\abs\xi} = \eu^{-(a+b)\abs\xi} = \varphi_{(a+b)X_1}(\xi)$ (independence and Exercise 23.10); injectivity identifies the laws.

(b) $\varphi_{aX_1+bX_2}(\xi) = \eu^{-a^2\xi^2/2} \eu^{-b^2\xi^2/2} = \eu^{-(a^2+b^2)\xi^2/2}$ : the law of $\sqrt{a^2+b^2}\,X_1$ .

(c) If $\varphi_X(\xi) = \eu^{-c\abs\xi^\alpha}$ , then $S_n = X_1 + \dots + X_n$ has $\varphi_{S_n} = \eu^{-cn\abs\xi^\alpha}$ , and $S_n/n^{1/\alpha}$ has $\varphi(\xi) = \eu^{-c\abs\xi^\alpha}$ again: exact self-reproduction under the $n^{1/\alpha}$ scaling — $\sqrt n$ for the Gaussian ( $\alpha = 2$ ), $n$ itself for Cauchy ( $\alpha = 1$ , Exercise 23.10(b)). A sum of i.i.d. variables can only converge (after affine normalization) to a law that is stable under such convolutions; the CLT says finite variance forces the Gaussian basin, and the Cauchy basin is reserved for laws with tails so heavy that $\E X^2 = \infty$ and even $\E\abs X = \infty$ — e.g. sums of Cauchy variables themselves. Universality has several islands, indexed by the tail exponent $\alpha \in \intoc02$ .

Exercise 23.12 ★★

(The empirical distribution function) Let $(X_n)$ be i.i.d. with distribution function $F$ , and $F_n(t) = \frac1n\#\{k \leq n : X_k \leq t\}$ . (a) Fix $t$ . Show that $n F_n(t) \sim \mathcal B(n, F(t))$ , that $F_n(t) \to F(t)$ a.s. (Theorem 22.13), and that

\sqrt n\,\bigl(F_n(t) - F(t)\bigr) \Longrightarrow \mathcal N\bigl(0,\ F(t)(1 - F(t))\bigr) .

(b) At which $t$ is the asymptotic variance maximal? Interpret: the median is where an empirical distribution is hardest to pin down. (c) For $F$ continuous, show that the law of $\sup_t\abs{F_n(t) - F(t)}$ does not depend on $F$ (reduce to uniform variables via Exercise 22.1) — the distribution-free miracle behind the Kolmogorov–Smirnov test; no computation of that law is asked.

Solution

Solution of Exercise 23.12.

(a) The indicators $\mathbf 1_{X_k \leq t}$ are i.i.d. Bernoulli of parameter $p = F(t)$ : their sum $nF_n(t)$ is binomial $\mathcal B(n, p)$ ; the strong law gives $F_n(t) \to p$ a.s., and the CLT (Theorem 23.8) applied to the same indicators (variance $p(1-p)$ ) gives the stated Gaussian limit.

(b) $p(1 - p)$ is maximal at $p = \frac12$ , i.e. where $F(t) = \frac12$ : at the median. Estimating tail probabilities is asymptotically easy (variance $\to 0$ as $p \to 0, 1$ ); the median region carries the largest statistical noise — the empirical curve wobbles most in its middle.

(c) For continuous $F$ , the variables $U_k = F(X_k)$ are i.i.d. uniform on $\intoo01$ (Exercise 22.1), and monotonicity of $F$ gives, writing $G_n$ for the empirical distribution function of the $U_k$ :

\sup_{t\in\R}\,\abs{F_n(t) - F(t)} = \sup_{u \in \operatorname{im}F}\,\abs{G_n(u) - u} = \sup_{u\in\intcc01}\abs{G_n(u) - u} :

the first equality because $\{X_k \leq t\} = \{U_k \leq F(t)\}$ up to null events (monotonicity; strict inequality can fail only on the flat parts of $F$ , where both sides are unchanged), and the second because a continuous $F$ , running from $0$ to $1$ , attains every value of $\intoo01$ (intermediate value theorem), and the endpoints add nothing ( $G_n(0) - 0 = 0$ and $G_n(1) - 1 = 0$ ). The right side involves only uniforms: one law for all $F$ — so a single table of critical values (that of Kolmogorov’s distribution) tests any continuous model against data.

23.6 Problem: Lindeberg’s proof of the CLT, with a rate

Problem 23.1

Weekend problem — the replacement method

Lindeberg (1922) proved the central limit theorem by an idea of disarming simplicity: swap the summands one at a time for Gaussians and control each swap by a Taylor expansion. The method needs no Fourier analysis, produces an explicit error rate, and today powers universality proofs across probability theory. Let $(X_i)$ be i.i.d., centered, $\V(X_1) = 1$ , with $\beta = \E\abs{X_1}^3 < \infty$ ; let $(N_i)$ be i.i.d. $\mathcal N(0,1)$ , independent of the $X_i$ (existence: Theorem 22.6). Set

T_n = \frac{X_1 + \dots + X_n}{\sqrt n}, \qquad G_n = \frac{N_1 + \dots + N_n}{\sqrt n} \sim \mathcal N(0,1).

Part I — The swapping identity. Fix $f \in \mathcal C^3_b(\R)$ (three bounded continuous derivatives; $M_3 = \sup\abs{f'''}$ ). For $0 \leq i \leq n$ define the hybrid sums

H_i = \frac{X_1 + \dots + X_i + N_{i+1} + \dots + N_n}{\sqrt n},

so $H_n = T_n$ and $H_0 = G_n$ .

Write $H_i = W_i + \frac{X_i}{\sqrt n}$ and $H_{i-1} = W_i + \frac{N_i}{\sqrt n}$ with $W_i = \frac{1}{\sqrt n}\bigl(\sum_{j<i}X_j + \sum_{j>i}N_j\bigr)$ , and note that $W_i$ is independent of the pair $(X_i, N_i)$ . Justify.
Taylor with integral or Lagrange remainder: for any real $w, h$ :

$\Bigl|f(w + h) - f(w) - f'(w)h - \tfrac12f''(w)h^2\Bigr| \leq \frac{M_3\,\abs h^3}{6} .$
Apply question 2 twice ( $h = \frac{X_i}{\sqrt n}$ and $h = \frac{N_i}{\sqrt n}$ at $w = W_i$ ), take expectations, and use independence plus the matching of the first two moments of $X_i$ and $N_i$ to show

$\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq \frac{M_3}{6}\cdot \frac{\beta + \gamma}{n^{3/2}}, \qquad \gamma = \E\abs{N_1}^3 = \frac{2\sqrt2}{\sqrt\pi} .$
Telescope over $i$ and conclude the Lindeberg bound:

$\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq \frac{M_3\,(\beta + \gamma)}{6\,\sqrt n} .$

Part II — From smooth $f$ to the CLT.

Show that $\E f(T_n) \to \E f(N)$ for every $f \in \mathcal C_b^3$ , and upgrade to all bounded continuous $f$ : given such $f$ and $\varepsilon$ , construct $f_\varepsilon \in \mathcal C^3_b$ with $\norm{f - f_\varepsilon}_\infty \leq \varepsilon$ on a large interval — e.g. convolve $f$ with a $\mathcal C^\infty$ bump (Theorem 12.9) — and handle the tails by tightness ( $\V(T_n) = 1$ and Chebyshev). Conclude $T_n \Rightarrow \mathcal N(0, 1)$ : the central limit theorem, re-proved.
Where did the proof use that the $X_i$ are identically distributed? Show that it barely did: state and prove the version for independent, centered, non-identical $X_i$ with $\sum_i\V(X_i) = s_n^2$ and third moments, obtaining the error $\frac{M_3}{6s_n^3}\sum_i\bigl(\E\abs{X_i}^3 + \V(X_i)^{3/2}\gamma\bigr)$ — Lindeberg’s true theorem in its Lyapunov form.

Part III — Quantitative dividends.

(Distribution functions) Let $t \in \R$ and approximate $\mathbf 1_{\intoc{-\infty}t}$ above and below by $\mathcal C^3_b$ ramps of width $\delta$ (construct them, with $M_3 = O(\delta^{-3})$ ). Combining with Part I, derive the two-term bound

$\sup_{t\in\R}\,\bigl|\P(T_n \leq t) - \Phi(t)\bigr| \;\leq\; \frac{C_1(\beta + \gamma)}{\delta^3\sqrt n} + C_2\,\delta \qquad (\text{every } \delta > 0),$

with explicit constants (the $C_2\delta$ term uses that $\Phi$ has density bounded by $\frac1{\sqrt{2\pi}}$ ), and optimize $\delta \sim n^{-1/8}$ to obtain a uniform rate of order $n^{-1/8}$ . (The optimal $n^{-1/2}$ — Berry–Esseen — needs finer tools; the point is an explicit rate from elementary swapping.)
(De Moivre–Laplace, quantified) Specialize to $X_i = 2B_i - 1$ (signs of fair coins): compare the conclusion with the local estimate of Problem 11.1, question 7 — what does each method give that the other does not?
(Universality) Explain in a paragraph why the replacement method shows more than the CLT: any statistic of the form $\E f(\text{sum})$ with $f$ smooth is insensitive, at order $n^{-1/2}$ , to the entire law of the summands beyond its first two moments — the “invariance principle” that underlies modern universality results (random matrices, random polynomials), of which the CLT is the first instance.

Part IV — Smoothing, pushed: better rates. The loss from $n^{-1/2}$ (smooth $f$ ) to $n^{-1/8}$ (distribution functions) came from charging $f'''$ in sup norm. The hybrids can repair part of it: they contain Gaussian summands, and Gaussians smooth.

(A hidden Gaussian) For $1 \leq i \leq n - 1$ , $h = \frac{X_i}{\sqrt n}$ or $\frac{N_i}{\sqrt n}$ , and $\theta \in \intcc01$ , write $W_i + \theta h = A + Z$ with $Z = \frac{N_{i+1} + \dots + N_n}{\sqrt n}$ . Show that $Z \sim \mathcal N\bigl(0, \frac{n-i}n\bigr)$ is independent of the pair $(A, h)$ , and deduce, for every continuous $g \in L^1(\R)$ ,

$\E\bigl[\abs h^3\,\abs{g(W_i + \theta h)}\bigr] \;\leq\; \sqrt{\frac{n}{2\pi(n - i)}}\; \norm{g}_{L^1}\;\E\abs h^3 .$
Combine question 10 with the integral form of the Taylor remainder,

$f(w + h) = f(w) + f'(w)h + \tfrac12f''(w)h^2 + \int_0^1\frac{(1 - \theta)^2}2\,f'''(w + \theta h)\,h^3\,\dd\theta,$

to redo questions 3–4: for $f \in \mathcal C^3_b$ with moreover $f''' \in L^1(\R)$ ,

$\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq \frac{\beta + \gamma}{3\sqrt{2\pi}}\cdot \frac{\norm{f'''}_{L^1}}{\sqrt n} + \frac{M_3(\beta + \gamma)}{6\,n^{3/2}}$

(question 10 handles the swaps $i \leq n - 1$ — use $\sum_{m=1}^{n-1}m^{-1/2} \leq 2\sqrt n$ — and question 3’s crude bound handles the last one). Check that question 7’s ramps satisfy $\norm{\psi_\delta'''}_{L^1} = K_1\delta^{-2}$ while $M_3 = K\delta^{-3}$ , feed them in, and optimize $\delta$ : the uniform distribution-function rate improves to $O(n^{-1/6})$ .
(Matching one more moment) Assume in addition $\E X_1^3 = 0$ and $\beta_4 = \E X_1^4 < \infty$ . Compute $\E N_1^3$ and $\E N_1^4$ , expand to fourth order, and prove along the same lines that the distribution-function rate becomes $O(n^{-1/4})$ (now $\norm{\psi_\delta^{(4)}}_{L^1} = K_2\delta^{-3}$ and $M_4 = K'\delta^{-4}$ ; choose $\delta = n^{-1/4}$ ).
(The obstruction) Suppose the first $k$ moments of $X_1$ agree with the Gaussian ones ( $k = 2$ always; $k = 3$ exactly when $\E X_1^3 = 0$ ; $k \geq 4$ essentially never, as $\E N_1^4 = 3$ ). Verify that the scheme of questions 10–12 delivers the distribution-function rate $n^{-(k-1)/(2k+2)}$ , by balancing $\delta^{-k}n^{-(k-1)/2}$ against $\delta$ , and observe that the exponent approaches the Berry–Esseen value $\frac12$ only as $k \to \infty$ . Explain in a few sentences why the swapping method saturates: each swap is charged in absolute value, whereas the Fourier route (Esseen’s smoothing inequality) exploits the oscillation of the characteristic-function difference and reaches $C\beta n^{-1/2}$ with three moments only.

Part V — Two dimensions: the multidimensional CLT, by swapping. Now let the $X_i$ be i.i.d. centered random vectors of $\R^2$ with covariance matrix $\Sigma$ and $\beta' = \E\norm{X_1}^3 < \infty$ (Euclidean norm).

(Gaussian vectors, to order) Diagonalize $\Sigma = PDP^{\mathsf T}$ (Exercise 20.8) and set $C = P\sqrt DP^{\mathsf T}$ . For $Z = (Z^1, Z^2)$ a pair of independent standard Gaussians (Theorem 22.6), show that $N = CZ$ is a Gaussian vector (Definition 23.10) of mean $0$ , covariance $\Sigma$ , with $\gamma' = \E\norm N^3 < \infty$ ; and that $G_n = \frac{N_1 + \dots + N_n}{\sqrt n}$ has law $\mathcal N(0, \Sigma)$ exactly for i.i.d. copies $N_i$ .
(Taylor in two variables) For $f \colon \R^2 \to \R$ of class $\mathcal C^3$ with $M_3 = \max_{\abs\alpha = 3}\sup\abs{\partial^\alpha f} < \infty$ , prove

$\Bigl|f(w + h) - f(w) - \langle\nabla f(w), h\rangle - \tfrac12\langle h, D^2f(w)\,h\rangle \Bigr| \leq \frac{M_3}6\,\bigl(\abs{h_1} + \abs{h_2}\bigr)^3 \leq \frac{\sqrt2\,M_3}3\, \norm h^3$

(study $t \mapsto f(w + th)$ on $\intcc01$ ).
(The CLT in $\R^2$ ) Run the replacement scheme on the vector hybrids $H_i$ : show that the first- and second-order terms cancel (means and covariances match), telescope, and upgrade as in question 5 (tightness from $\E\norm{T_n}^2 = \operatorname{tr}\Sigma$ ; mollification now in $\R^2$ , Theorem 12.9) to conclude: for every bounded continuous $f \colon \R^2 \to \R$ ,

$\E\,f\Bigl(\frac{X_1 + \dots + X_n}{\sqrt n}\Bigr) \longrightarrow \E\,f(N), \qquad N \sim \mathcal N(0, \Sigma) :$

Theorem 23.12 in dimension $2$ , with a rate for smooth $f$ and no Fourier analysis.
(Cramér–Wold, and a joint fluctuation) Deduce that $\langle t, \frac{S_n}{\sqrt n}\rangle \Rightarrow \mathcal N(0, t^{\mathsf T}\Sigma t)$ for every fixed $t \in \R^2$ . Application: for i.i.d. real $(\xi_i)$ , centered, $\E\xi_1^2 = 1$ , $\E\xi_1^6 < \infty$ (so that Part V applies to $V_i = (\xi_i, \xi_i^2 - 1)$ ), show

$\frac1{\sqrt n}\Bigl(\sum_{i\leq n}\xi_i,\ \sum_{i\leq n}(\xi_i^2 - 1)\Bigr) \Longrightarrow \mathcal N\Bigl(0, \begin{pmatrix} 1 & \E\xi_1^3\\ \E\xi_1^3 & \E\xi_1^4 - 1\end{pmatrix}\Bigr) :$

empirical mean and empirical second moment fluctuate jointly Gaussianly — independently in the limit if and only if $\E\xi_1^3 = 0$ (Theorem 23.11).

Part VI — The delta method.

Let $(\hat\theta_n)$ be random variables with $\sqrt n(\hat\theta_n - \theta) \Rightarrow \mathcal N(0, \sigma^2)$ for a real parameter $\theta$ , and let $g$ be differentiable at $\theta$ . Prove the delta method:

$\sqrt n\bigl(g(\hat\theta_n) - g(\theta)\bigr) \Longrightarrow \mathcal N\bigl(0, g'(\theta)^2\sigma^2\bigr)$

(write $g(x) - g(\theta) = (g'(\theta) + \eta(x))(x - \theta)$ with $\eta \to 0$ at $\theta$ ; show $\hat\theta_n \to \theta$ , then $\eta(\hat\theta_n) \to 0$ , in probability; finish with Slutsky, Exercise 23.8, and Exercise 23.4(b)).
Applications. (a) For i.i.d. real $(\xi_i)$ with mean $\mu$ and variance $\sigma^2$ , and $\bar X_n = \frac1n\sum_{i\leq n}\xi_i$ : show $\sqrt n(\bar X_n^2 - \mu^2) \Rightarrow \mathcal N(0, 4\mu^2\sigma^2)$ when $\mu \neq 0$ , and that for $\mu = 0$ the correct statement lives at another scale: $n\bar X_n^2 \Rightarrow \sigma^2N^2$ with $N \sim \mathcal N(0,1)$ (identify the limit’s distribution function). (b) (Variance stabilization) For $\hat p_n$ the success frequency of a $\mathcal B(1, p)$ sample, $p \in \intoo01$ : show that $g(p) = \arcsin\sqrt p$ satisfies

$\sqrt n\,\bigl(g(\hat p_n) - g(p)\bigr) \Longrightarrow \mathcal N\Bigl(0, \frac14\Bigr)$

whatever $p$ — an asymptotic error bar free of the unknown parameter; compare with Example 23.9.

Part VII — Poisson, by the same method: Le Cam’s theorem. Replacement knows a second universality class: sums of many independent rare events. For laws on $\N$ the right distance is total variation,

d_{\mathrm{TV}}(\mu, \nu) = \sup_{A\subseteq\N}\, \abs{\mu(A) - \nu(A)} .

Show that $d_{\mathrm{TV}}(\mu, \nu) = \frac12\sum_{k\geq0}\abs{\mu(\{k\}) - \nu(\{k\})}$ , and prove the coupling bound: for any pair $(X, Y)$ of random variables with laws $\mu$ and $\nu$ on the same space, $d_{\mathrm{TV}}(\mu, \nu) \leq \P(X \neq Y)$ .
Compute exactly, for $p \in \intoo01$ :

$d_{\mathrm{TV}}\bigl(\mathcal B(1, p), \mathcal P(p)\bigr) = p\bigl(1 - \eu^{-p}\bigr) \leq p^2 .$
(Le Cam, by swapping) Let $X_i \sim \mathcal B(1, p_i)$ and $Y_i \sim \mathcal P(p_i)$ , the $2n$ variables independent; $S = X_1 + \dots + X_n$ , and recall $Y_1 + \dots + Y_n \sim \mathcal P(\lambda)$ with $\lambda = \sum_ip_i$ (Exercise 23.1). Swap one coordinate at a time in the integer hybrids $H_i = Y_1 + \dots + Y_i + X_{i+1} + \dots + X_n$ : show, for every $A \subseteq \N$ ,

$\abs{\P(H_{i-1} \in A) - \P(H_i \in A)} \leq d_{\mathrm{TV}}\bigl(\mathcal B(1, p_i), \mathcal P(p_i)\bigr),$

and conclude Le Cam’s inequality:

$d_{\mathrm{TV}}\bigl(\text{law of } S,\ \mathcal P(\lambda)\bigr) \leq \sum_{i=1}^np_i^2 .$
Dividends. (a) For $p_i = \frac\lambda n$ : the bound is $\frac{\lambda^2}n$ — the law of rare events (Exercise 23.5) upgraded to an explicit rate, uniform over all events, and valid for unequal $p_i$ as well. (b) $500$ letters are delivered, each going astray independently with probability $\frac1{500}$ : bound the error of the Poisson model of parameter $1$ , and estimate the probability that no letter goes astray. (c) Close the problem: compare the two universality classes met here — Gaussian (many small spread-out contributions; two moments matched; Taylor) and Poisson (many rare contributions; one mean matched; an exact total-variation coupling) — and the single replacement method behind both.
(Relative error and the log transform) Let $(X_n)$ be i.i.d., positive, mean $\mu > 0$ , variance $\sigma^2$ , and $\bar X_n$ the empirical mean. Show by the delta method that

$\sqrt n\,\bigl(\ln\bar X_n - \ln\mu\bigr) \Longrightarrow \mathcal N\Bigl(0,\ \frac{\sigma^2}{\mu^2}\Bigr) :$

the asymptotic parameter of $\ln\bar X_n$ is the coefficient of variation $\sigma/\mu$ — relative, scale-free error. Deduce a $95\%$ confidence interval for $\mu$ of the multiplicative form $\bar X_n\cdot\eu^{\pm1.96\,\sigma/(\mu\sqrt n)}$ , and explain when it is preferable to the additive one.
(The third moment steers the error) For $X \sim$ Bernoulli( $p$ ) centered, compute $\E\bigl[(X - p)^3\bigr] = p(1-p)(1-2p)$ . Using Part IV’s analysis (the swap error is driven by third moments), explain why the normal approximation of $\mathcal B(n, p)$ is asymmetric for $p \neq \frac12$ — overshooting on one side, undershooting on the other — and why $p = \frac12$ enjoys the faster matched-moment rate. Verify the sign of the skew numerically on $\mathcal B(20, 0.1)$ against $\mathcal N(2, 1.8)$ : compare $\P(S = 0) = 0.9^{20}$ with the Gaussian mass of $\intoo{-\infty}{0.5}$ .

Solution

Solution of Problem 23.1.

1. The family $(X_1, \dots, X_n, N_1, \dots, N_n)$ is independent: the two blocks are independent of each other by construction and each block is i.i.d. $W_i$ is a measurable function of the variables $(X_j)_{j<i}$ and $(N_j)_{j>i}$ only, all distinct from $X_i$ and $N_i$ : by the coalition principle (Theorem 22.5), $W_i$ is independent of the pair $(X_i, N_i)$ . The decompositions $H_i = W_i + \frac{X_i}{\sqrt n}$ and $H_{i-1} = W_i + \frac{N_i}{\sqrt n}$ are immediate from the definitions: passing from $H_i$ to $H_{i-1}$ swaps the single summand $X_i$ for $N_i$ .

2. Taylor–Lagrange at order $3$ : there is $c$ between $w$ and $w + h$ with $f(w + h) = f(w) + f'(w)h + \frac12f''(w)h^2 + \frac16f'''(c)h^3$ , and $\abs{f'''(c)} \leq M_3$ gives the bound.

3. Subtracting the two expansions at the common base point $w = W_i$ :

f(H_i) - f(H_{i-1}) = f'(W_i)\,\frac{X_i - N_i}{\sqrt n} + \frac{f''(W_i)}{2}\,\frac{X_i^2 - N_i^2}{n} + R_i, \qquad \abs{R_i} \leq \frac{M_3}{6}\cdot \frac{\abs{X_i}^3 + \abs{N_i}^3}{n^{3/2}} .

Take expectations. By question 1, $f'(W_i)$ and $f''(W_i)$ are independent of $(X_i, N_i)$ , so the mixed expectations factor:

\begin{align*} \E\Bigl[f'(W_i)\,\frac{X_i - N_i}{\sqrt n}\Bigr] &= \E\bigl[f'(W_i)\bigr]\,\frac{\E X_i - \E N_i}{\sqrt n} = 0, \\ \E\Bigl[f''(W_i)\,\frac{X_i^2 - N_i^2}{n}\Bigr] &= \E\bigl[f''(W_i)\bigr]\,\frac{1 - 1}{n} = 0 : \end{align*}

the first two moments of $X_i$ and $N_i$ match, and only the remainder survives:

\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq \E\abs{R_i} \leq \frac{M_3}{6}\cdot\frac{\beta + \gamma}{n^{3/2}} .

The Gaussian third moment: $\gamma = \E\abs{N_1}^3 = 2\int_0^\infty x^3\,\frac{\eu^{-x^2/2}}{\sqrt{2\pi}}\,\dd x = \frac{2}{\sqrt{2\pi}}\int_0^\infty 2u\,\eu^{-u}\dd u = \frac{4}{\sqrt{2\pi}} = \frac{2\sqrt2}{\sqrt\pi}$ (substitution $u = x^2/2$ , then $\Gamma(2) = 1$ ).

4. Telescoping $\E f(T_n) - \E f(G_n) = \sum_{i=1}^n\bigl(\E f(H_i) - \E f(H_{i-1})\bigr)$ and applying question 3 to each of the $n$ terms:

\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq n \cdot \frac{M_3(\beta + \gamma)}{6\,n^{3/2}} = \frac{M_3\,(\beta + \gamma)}{6\,\sqrt n} .

5. $G_n$ is $\mathcal N(0,1)$ exactly for every $n$ (a normalized sum of independent standard Gaussians, Exercise 23.3), so $\E f(G_n) = \E f(N)$ and question 4 reads $\abs{\E f(T_n) - \E f(N)} \leq \frac{M_3(\beta+\gamma)}{6\sqrt n} \to 0$ for $f \in \mathcal C^3_b$ . Upgrade. Let $f$ be bounded continuous, $M = \sup\abs f$ , $\varepsilon > 0$ . Choose $A \geq 1$ with $\frac1{A^2} \leq \varepsilon$ : Chebyshev with $\V(T_n) = 1$ gives $\P(\abs{T_n} > A) \leq \varepsilon$ for all $n$ , and likewise $\P(\abs N > A) \leq \varepsilon$ . Let $\chi$ be $\mathcal C^\infty$ with $\mathbf 1_{\intcc{-A}A} \leq \chi \leq \mathbf 1_{\intcc{-A-1}{A+1}}$ (a smooth plateau, built by mollifying $\mathbf 1_{\intcc{-A-\frac12}{A+\frac12}}$ , Theorem 12.9); $g = f\chi$ is continuous with compact support, hence uniformly continuous, so its mollification $g_\eta = g * \rho_\eta$ is $\mathcal C^\infty$ with bounded derivatives of all orders and $\norm{g - g_\eta}_\infty \leq \varepsilon$ for $\eta$ small enough. For $T = T_n$ or $N$ , since $f = g$ on $\intcc{-A}A$ and $\abs{f - g} \leq 2M$ everywhere:

\bigl|\E f(T) - \E g_\eta(T)\bigr| \leq \E\abs{(f - g)(T)} + \norm{g - g_\eta}_\infty \leq 2M\,\P(\abs T > A) + \varepsilon \leq (2M + 1)\,\varepsilon .

Combining with $\E g_\eta(T_n) \to \E g_\eta(N)$ (question 4 applies: $g_\eta \in \mathcal C^3_b$ ):

\limsup_n\;\bigl|\E f(T_n) - \E f(N)\bigr| \leq 2(2M + 1)\,\varepsilon ,

and $\varepsilon$ was arbitrary: $\E f(T_n) \to \E f(N)$ for every bounded continuous $f$ , i.e. $T_n \Rightarrow \mathcal N(0,1)$ .

6. Identical distribution entered only through one sentence: “ $X_i$ and $N_i$ have the same first two moments”. So let $X_1, \dots, X_n$ be independent, centered, with variances $\sigma_i^2$ and finite third moments, $s_n^2 = \sum_i\sigma_i^2 > 0$ , and take $N_i \sim \mathcal N(0, \sigma_i^2)$ independent of everything. Define the hybrids with normalization $s_n$ : $H_i = \frac1{s_n}(\sum_{j\leq i}X_j + \sum_{j>i}N_j)$ . In the $i$ -th swap, $\E X_i = \E N_i = 0$ and $\E X_i^2 = \E N_i^2 = \sigma_i^2$ again kill the $f'$ and $f''$ terms, and the remainder gives (using $\E\abs{N_i}^3 = \sigma_i^3\gamma$ by scaling):

\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq \frac{M_3}{6\,s_n^3}\bigl(\E\abs{X_i}^3 + \sigma_i^3\gamma\bigr) .

Telescoping:

\Bigl|\E f\Bigl(\frac{X_1 + \dots + X_n}{s_n}\Bigr) - \E f(N)\Bigr| \leq \frac{M_3}{6\,s_n^3}\sum_{i=1}^n \Bigl(\E\abs{X_i}^3 + \V(X_i)^{3/2}\,\gamma\Bigr) .

Since $\sigma_i^3 = (\E X_i^2)^{3/2} \leq \E\abs{X_i}^3$ (the power–mean inequality, i.e. Jensen for $t \mapsto t^{3/2}$ applied to $X_i^2$ ), the right side is at most $\frac{M_3(1 + \gamma)}{6}\cdot \frac{\sum_i\E\abs{X_i}^3}{s_n^3}$ : under Lyapunov’s condition $\frac1{s_n^3}\sum_i\E\abs{X_i}^3 \to 0$ , the normalized sums converge in law to $\mathcal N(0,1)$ — the CLT without identical distribution.

7. Let $\rho \in \mathcal C^\infty_c(\intoo01)$ with $\int\rho = 1$ and set $\psi(x) = \int_x^1\rho(s)\dd s$ : $\psi$ is $\mathcal C^\infty$ , nonincreasing, $\psi = 1$ on $\R_-$ , $\psi = 0$ on $\intco1\infty$ ; let $K = \norm{\psi'''}_\infty$ . For $t \in \R$ and $\delta > 0$ define $\psi_\delta(x) = \psi\bigl(\frac{x - t}\delta\bigr)$ and $\tilde\psi_\delta(x) = \psi\bigl(\frac{x - t}\delta + 1\bigr)$ : these are $\mathcal C^3_b$ with third derivative bounded by $K/\delta^3$ , and

\mathbf 1_{\intoc{-\infty}{t-\delta}} \leq \tilde\psi_\delta \leq \mathbf 1_{\intoc{-\infty}t} \leq \psi_\delta \leq \mathbf 1_{\intoc{-\infty}{t+\delta}} .

Upper bound: by question 4 applied to $\psi_\delta$ (with $M_3 = K/\delta^3$ ),

\P(T_n \leq t) \leq \E\psi_\delta(T_n) \leq \E\psi_\delta(N) + \frac{K(\beta + \gamma)}{6\,\delta^3\sqrt n} \leq \Phi(t + \delta) + \frac{K(\beta + \gamma)}{6\,\delta^3\sqrt n} \leq \Phi(t) + \frac{\delta}{\sqrt{2\pi}} + \frac{K(\beta + \gamma)}{6\,\delta^3\sqrt n},

because $\Phi$ is Lipschitz with constant $\frac1{\sqrt{2\pi}}$ (its density is bounded by $\frac1{\sqrt{2\pi}}$ ). The symmetric lower bound via $\tilde\psi_\delta$ gives the two-term estimate

\sup_{t\in\R}\,\bigl|\P(T_n \leq t) - \Phi(t)\bigr| \leq \frac{K(\beta + \gamma)}{6}\cdot\frac{1}{\delta^3\sqrt n} + \frac{\delta}{\sqrt{2\pi}} \qquad(\delta > 0\ \text{arbitrary}).

The two terms balance when $\delta^{-3}n^{-1/2} \asymp \delta$ , i.e. $\delta = n^{-1/8}$ : both are then $O(n^{-1/8})$ , an explicit uniform rate valid for every $n$ . (The optimal Berry–Esseen rate $C\beta/\sqrt n$ requires the Fourier smoothing inequality; swapping trades sharpness for complete elementarity.)

8. For $X_i = 2B_i - 1$ (fair signs): centered, variance $1$ , and $\abs{X_i} = 1$ so $\beta = 1$ . Question 7 then bounds $\sup_t\abs{\P(\frac{S_n}{\sqrt n} \leq t) - \Phi(t)}$ explicitly and uniformly for every finite $n$ — a global, non-asymptotic statement about the distribution function. The local estimate of Problem 11.1, question 7, gives instead the exact asymptotics of an individual atom, $\P(S_{2n} = 2k) \sim \frac{\eu^{-k^2/n}}{\sqrt{\pi n}}$ : it resolves probabilities of size $n^{-1/2}$ , far below question 7’s $n^{-1/8}$ resolution, but it is pointwise, asymptotic (no explicit error at fixed $n$ ) and tied to this particular lattice law. Local precision versus global uniformity: the two methods are complementary, and summing the local estimate over $k \in \intint{a\sqrt n}{b\sqrt n}$ recovers de Moivre–Laplace on intervals — with a sharper rate, but only for this law.

9. The swapping argument used nothing about the law of the $X_i$ beyond $\E X_i = 0$ , $\E X_i^2 = 1$ and the finiteness of $\E\abs{X_i}^3$ : had we replaced the Gaussians $N_i$ by any other i.i.d. family with the same first two moments and finite third moment, the same telescoping would bound $\abs{\E f(\text{sum}_X) - \E f(\text{sum}_Y)}$ by $O(n^{-1/2})$ for every smooth $f$ . Smooth statistics of large independent sums are therefore universal: up to a quantified error, they depend on the summands’ law only through two numbers. This is the invariance principle: prove a limit theorem for the most computable law (the Gaussian, where everything is exact), then transfer it to all laws by swapping. The same scheme — with sums replaced by more elaborate functionals — drives Wigner’s semicircle law for random matrices, the universality of roots of random polynomials, and much of modern probability; the central limit theorem is its first and simplest instance.

10. $Z$ is a Borel function of $(N_{i+1}, \dots, N_n)$ only, while $A = W_i + \theta h - Z$ and $h$ are functions of the remaining variables of the independent family $(X_1, \dots, X_n, N_1, \dots, N_n)$ : by the coalition principle (Theorem 22.5), $Z$ is independent of $(A, h)$ . As a sum of the independent $N_j/\sqrt n \sim \mathcal N(0, \frac1n)$ , $Z \sim \mathcal N(0, s^2)$ with $s^2 = \frac{n-i}n$ (Exercise 23.3), with density bounded by $\frac1{s\sqrt{2\pi}}$ . The law of $((A, h), Z)$ is the product of the two marginal laws, so Tonelli (transfer) freezes the first block: with $G(a) = \E\abs{g(a + Z)} = \int\abs{g(a + z)}\,\varphi_s(z)\,\dd z \leq \frac{\norm g_{L^1}}{s\sqrt{2\pi}}$ for every $a$ ,

\E\bigl[\abs h^3\abs{g(A + Z)}\bigr] = \E\bigl[\abs h^3\,G(A)\bigr] \leq \frac{\norm g_{L^1}}{s\sqrt{2\pi}}\,\E\abs h^3 = \sqrt{\frac{n}{2\pi(n-i)}}\,\norm g_{L^1}\,\E\abs h^3 .

11. The integral form of Taylor’s formula follows by integrating $f(w + h) - f(w) = h\int_0^1f'(w + \theta h)\,\dd\theta$ by parts twice in $\theta$ . Taking expectations in the $i$ -th swap, the orders $0, 1, 2$ cancel exactly as in question 3, and the two remainders (for $h = X_i/\sqrt n$ and $N_i/\sqrt n$ ) are bounded, for $i \leq n - 1$ , by question 10 with $g = f'''$ :

\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq \int_0^1\frac{(1-\theta)^2}2\,\dd\theta\; \sqrt{\frac{n}{2\pi(n-i)}}\,\norm{f'''}_{L^1} \frac{\beta + \gamma}{n^{3/2}} = \frac{\beta + \gamma}{6\,n^{3/2}} \sqrt{\frac{n}{2\pi(n-i)}}\,\norm{f'''}_{L^1} .

Summing, with $\sum_{i=1}^{n-1}\sqrt{\frac n{n-i}} = \sqrt n\sum_{m=1}^{n-1}m^{-1/2} \leq 2n$ , and adding question 3’s bound for the last swap ( $i = n$ , no Gaussian left):

\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq \frac{\beta + \gamma}{3\sqrt{2\pi}}\cdot \frac{\norm{f'''}_{L^1}}{\sqrt n} + \frac{M_3(\beta + \gamma)}{6\,n^{3/2}} .

Ramps: $\psi_\delta'''(x) = \delta^{-3}\psi'''\bigl(\frac{x - t}\delta\bigr)$ , so $M_3 = K\delta^{-3}$ with $K = \norm{\psi'''}_\infty$ and $\norm{\psi_\delta'''}_{L^1} = \delta^{-2} \norm{\psi'''}_{L^1} = K_1\delta^{-2}$ (substitution). The sandwich of question 7 then gives

\sup_t\,\bigl|\P(T_n \leq t) - \Phi(t)\bigr| \leq \frac{K_1(\beta + \gamma)}{3\sqrt{2\pi}}\cdot \frac1{\delta^2\sqrt n} + \frac{K(\beta + \gamma)}{6}\cdot \frac1{\delta^3n^{3/2}} + \frac\delta{\sqrt{2\pi}} .

At $\delta = n^{-1/6}$ the first and third terms are $O(n^{-1/6})$ and the middle one $O(n^{-1})$ : a uniform rate $O(n^{-1/6})$ , strictly better than question 7’s $n^{-1/8}$ — the Gaussian half of the hybrid did the extra smoothing.

12. $\E N_1^3 = 0$ (odd integrand), and integration by parts gives $\E N_1^4 = 3\,\E N_1^2 = 3$ ( $\int x^3\cdot x\varphi(x)\dd x = 3\int x^2\varphi$ ). For $f$ of class $\mathcal C^4$ with bounded derivatives, expand each swap to fourth order: the third-order terms carry the factor $\E X_i^3 - \E N_i^3 = 0$ (independence factors them as in question 3), so only the fourth-order remainder $\int_0^1\frac{(1-\theta)^3}6f^{(4)}(w + \theta h)h^4\dd\theta$ survives, with $\int_0^1 \frac{(1-\theta)^3}6\dd\theta = \frac1{24}$ and $\E h^4 = \beta_4n^{-2}$ or $3n^{-2}$ . Question 10 (with $g = f^{(4)}$ ) bounds the swaps $i \leq n - 1$ , and summing as in question 11:

\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq \frac{\beta_4 + 3}{12\sqrt{2\pi}}\cdot \frac{\norm{f^{(4)}}_{L^1}}{n} + \frac{M_4(\beta_4 + 3)}{24\,n^2} .

With $\norm{\psi_\delta^{(4)}}_{L^1} = K_2\delta^{-3}$ and $M_4 = K'\delta^{-4}$ , the distribution-function bound becomes $C\delta^{-3}n^{-1} + C'\delta^{-4}n^{-2} + \frac\delta{\sqrt{2\pi}}$ ; at $\delta = n^{-1/4}$ the outer terms are $O(n^{-1/4})$ and the middle $O(n^{-1})$ : rate $O(n^{-1/4})$ .

13. With $k$ matched moments the surviving remainder per swap is of order $\E\abs h^{k+1} \asymp n^{-(k+1)/2}$ ; the hidden-Gaussian bound charges $\norm{f^{(k+1)}}_{L^1}$ and the sum over swaps contributes the factor $2n$ , giving $\asymp\norm{f^{(k+1)}}_{L^1}\, n^{-(k-1)/2}$ for smooth $f$ . Ramps cost $\norm{\psi_\delta^{(k+1)}}_{L^1} \asymp \delta^{-k}$ , so the distribution-function error is $\asymp \delta^{-k}n^{-(k-1)/2} + \delta$ , balanced at $\delta = n^{-(k-1)/(2k+2)}$ : rate $n^{-(k-1)/(2k+2)}$ , which is $n^{-1/6}$ for $k = 2$ , $n^{-1/4}$ for $k = 3$ , and tends to $n^{-1/2}$ only as $k \to \infty$ — but $k \geq 4$ would force $\E X_1^4 = 3$ and beyond, i.e. a law that already imitates the Gaussian. The saturation is structural: swapping adds $n$ swap errors in absolute value, renouncing all cancellation between swaps. The Fourier proof compares characteristic functions, where the errors appear with their oscillating phases; Esseen’s smoothing inequality converts $\abs{\varphi_{T_n} - \varphi_N}$ , integrated against $\frac{\dd\xi}{\abs\xi}$ , into a distribution-function bound at only logarithmic cost, and delivers Berry–Esseen’s $C\beta n^{-1/2}$ from three moments. Replacement trades optimality for robustness — and, as Part VII shows, for portability.

14. $\Sigma$ is symmetric positive semidefinite; with $\Sigma = PDP^{\mathsf T}$ ( $P$ orthogonal, $D \geq 0$ diagonal, Exercise 20.8), the symmetric $C = P\sqrt DP^{\mathsf T}$ satisfies $C^2 = \Sigma$ . For any $t \in \R^2$ , $\langle t, CZ\rangle = \langle Ct, Z\rangle = (Ct)_1Z^1 + (Ct)_2Z^2$ is a linear combination of independent Gaussians, hence Gaussian (Exercise 23.3): $N = CZ$ is a Gaussian vector; its mean is $0$ and its covariance $\E[NN^{\mathsf T}] = C\,\E[ZZ^{\mathsf T}]\,C^{\mathsf T} = CC^{\mathsf T} = \Sigma$ . Moments: $\norm N^3 \leq (\abs{N_1} + \abs{N_2})^3 \leq 4(\abs{N_1}^3 + \abs{N_2}^3)$ (convexity of $x^3$ on $\R_+$ ), and each coordinate is a real Gaussian with moments of all orders (Exercise 11.10): $\gamma' < \infty$ . Finally each $\langle t, G_n\rangle = \frac1{\sqrt n}\sum_i\langle t, N_i\rangle$ is a normalized sum of i.i.d. $\mathcal N(0, t^{\mathsf T}\Sigma t)$ , hence exactly $\mathcal N(0, t^{\mathsf T}\Sigma t)$ : $G_n$ is a Gaussian vector with mean $0$ and covariance $\Sigma$ , and its law is $\mathcal N(0, \Sigma)$ (Definition 23.10: the law is determined by these data).

15. Let $\phi(t) = f(w + th)$ , $t \in \intcc01$ : $\phi$ is $\mathcal C^3$ with

\phi'''(t) = \sum_{j,k,l\in\{1,2\}}\partial_{jkl}f(w + th)\,h_jh_kh_l, \qquad \abs{\phi'''(t)} \leq M_3\Bigl(\sum_j\abs{h_j}\Bigr)^3 = M_3(\abs{h_1} + \abs{h_2})^3 .

Taylor–Lagrange at order $3$ for $\phi$ between $0$ and $1$ gives the first inequality; Cauchy–Schwarz gives $\abs{h_1} + \abs{h_2} \leq \sqrt2\norm h$ , whence the constant $\frac{2\sqrt2M_3}6 = \frac{\sqrt2M_3}3$ .

16. Define $H_i$ and $W_i$ as in question 1, now in $\R^2$ ; the coalition argument is unchanged. In the $i$ -th swap, the first-order terms give $\sum_j\E[\partial_jf(W_i)]\,(\E X_{i,j} - \E N_{i,j})/ \sqrt n = 0$ and the second-order terms give $\frac1{2n}\sum_{j,k}\E[\partial_{jk}f(W_i)]\,(\Sigma_{jk} - \Sigma_{jk}) = 0$ : means and covariances match. Question 15 bounds the two remainders:

\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq \frac{\sqrt2M_3}3\cdot\frac{\E\norm{X_i}^3 + \E\norm{N_i}^3}{n^{3/2}} = \frac{\sqrt2M_3(\beta' + \gamma')}{3\,n^{3/2}},

and telescoping over the $n$ swaps:

\Bigl|\E f\Bigl(\frac{S_n}{\sqrt n}\Bigr) - \E f(G_n)\Bigr| \leq \frac{\sqrt2\,M_3(\beta' + \gamma')}{3\sqrt n}, \qquad G_n \sim \mathcal N(0, \Sigma)\ \text{exactly} .

Upgrade: $\E\norm{T_n}^2 = \E\norm{X_1}^2 = \operatorname{tr}\Sigma$ (cross terms vanish by independence and centering), so $\P(\norm{T_n} > A) \leq \operatorname{tr}\Sigma/A^2$ , and likewise for $N$ : tightness. Given a bounded continuous $f$ and $\varepsilon > 0$ , multiply by a smooth plateau $\chi$ equal to $1$ on the ball of radius $A$ and supported in radius $A + 1$ (mollify an indicator in $\R^2$ , Theorem 12.9); $g = f\chi$ is uniformly continuous with compact support, so its two-dimensional mollification $g_\eta$ is $\mathcal C^\infty$ with bounded derivatives of all orders and $\norm{g - g_\eta}_\infty \leq \varepsilon$ for small $\eta$ . The three- $\varepsilon$ chain of question 5 then transfers verbatim: $\E f(T_n) \to \E f(N)$ for every bounded continuous $f \colon \R^2 \to \R$ . This is Theorem 23.12 for $d = 2$ , now proved — swapping sidesteps the two-dimensional Lévy theorem that the chapter had left admitted.

17. For bounded continuous $g \colon \R \to \R$ , the map $x \mapsto g(\langle t, x\rangle)$ is bounded continuous on $\R^2$ , so question 16 gives $\E g(\langle t, T_n\rangle) \to \E g(\langle t, N\rangle)$ : every projection converges in distribution, and $\langle t, N\rangle \sim \mathcal N(0, t^{\mathsf T}\Sigma t)$ . (This is the easy direction of Cramér–Wold: joint convergence implies convergence of all linear images.) Application: $V_i = (\xi_i, \xi_i^2 - 1)$ are i.i.d. centered vectors ( $\E\xi_1^2 = 1$ ), with covariance entries $\V(\xi_1) = 1$ , $\operatorname{Cov}(\xi_1, \xi_1^2 - 1) = \E\xi_1^3$ and $\V(\xi_1^2 - 1) = \E\xi_1^4 - 1$ ; the third moment $\E\norm{V_1}^3 \leq 4\bigl(\E\abs{\xi_1}^3 + \E\abs{\xi_1^2 - 1}^3\bigr)$ is finite when $\xi_1 \in L^6$ . Question 16 yields the displayed joint Gaussian limit, and Theorem 23.11(2): the two limit coordinates are independent exactly when the covariance $\E\xi_1^3$ vanishes — for symmetric laws, empirical mean and empirical variance decouple asymptotically.

18. Write $g(x) - g(\theta) = (g'(\theta) + \eta(x))(x - \theta)$ where $\eta(x) = \frac{g(x) - g(\theta)}{x - \theta} - g'(\theta)$ for $x \neq \theta$ and $\eta(\theta) = 0$ : differentiability at $\theta$ means precisely $\eta(x) \to 0$ as $x \to \theta$ . Step 1: $\hat\theta_n \to \theta$ in probability: for $\varepsilon > 0$ and any $A > 0$ , eventually $\varepsilon\sqrt n \geq A$ , so $\P(\abs{\hat\theta_n - \theta} > \varepsilon) \leq \P(\abs{\sqrt n(\hat\theta_n - \theta)} > A) \to \P(\sigma\abs N > A)$ (distribution functions converge at the continuity points $\pm A$ ), and the right side tends to $0$ as $A \to \infty$ . Step 2: $\eta(\hat\theta_n) \to 0$ in probability: given $\varepsilon' > 0$ , pick $\delta$ with $\abs\eta \leq \varepsilon'$ on $\abs{x - \theta} \leq \delta$ ; then $\P(\abs{\eta(\hat\theta_n)} > \varepsilon') \leq \P(\abs{\hat\theta_n - \theta} > \delta) \to 0$ . Step 3:

\sqrt n\bigl(g(\hat\theta_n) - g(\theta)\bigr) = g'(\theta)\,\sqrt n(\hat\theta_n - \theta) + \eta(\hat\theta_n)\cdot\sqrt n(\hat\theta_n - \theta) .

By Slutsky’s product rule (Exercise 23.8, with the sequence $\eta(\hat\theta_n) \to 0$ in probability and the convergent-in-law $\sqrt n(\hat\theta_n - \theta)$ ), the second term converges in law to $0\cdot\mathcal N(0, \sigma^2) = 0$ , hence to $0$ in probability (Exercise 23.4(b)); the first converges in law to $g'(\theta)\mathcal N(0, \sigma^2)$ (Slutsky again, or the affine rule for characteristic functions); Slutsky’s sum rule assembles them: the limit is $\mathcal N(0, g'(\theta)^2\sigma^2)$ .

19. (a) The CLT gives $\sqrt n(\bar X_n - \mu) \Rightarrow \mathcal N(0, \sigma^2)$ ; the delta method with $g(x) = x^2$ , $g'(\mu) = 2\mu$ , gives $\sqrt n(\bar X_n^2 - \mu^2) \Rightarrow \mathcal N(0, 4\mu^2\sigma^2)$ — degenerate (limit $0$ ) when $\mu = 0$ . In that case the fluctuation lives one scale up: $n\bar X_n^2 = (\sqrt n\,\bar X_n)^2$ , and for $t > 0$

\P\bigl(n\bar X_n^2 \leq t\bigr) = \P\bigl(-\sqrt t \leq \sqrt n\,\bar X_n \leq \sqrt t\bigr) \longrightarrow \Phi\Bigl(\frac{\sqrt t}\sigma\Bigr) - \Phi\Bigl(-\frac{\sqrt t}\sigma\Bigr) = \P(\sigma^2N^2 \leq t) :

$n\bar X_n^2 \Rightarrow \sigma^2N^2$ , the square of a Gaussian (a “chi-squared” law) — when the first derivative dies, the second-order term of Taylor dictates a non-Gaussian limit. (b) Here $\sqrt n(\hat p_n - p) \Rightarrow \mathcal N(0, p(1 - p))$ and $g(p) = \arcsin\sqrt p$ has $g'(p) = \frac1{2\sqrt{p(1 - p)}}$ , so $g'(p)^2\,p(1 - p) = \frac14$ : the limit is $\mathcal N(0, \frac14)$ for every $p \in \intoo01$ . On the $\arcsin$ scale the asymptotic $95\%$ error bar is $\pm \frac{0.98}{\sqrt n}$ , known in advance — whereas in Example 23.9 the width involved the unknown $\sigma = \sqrt{p(1-p)}$ , to be worst-cased by $\frac12$ or estimated: the transformation stabilizes the variance.

20. Let $A^* = \{k : \mu(\{k\}) > \nu(\{k\})\}$ and $\Delta_k = \mu(\{k\}) - \nu(\{k\})$ , so $\sum_k\Delta_k = 0$ . For any $A \subseteq \N$ : $\mu(A) - \nu(A) = \sum_{k\in A}\Delta_k \leq \sum_{k\in A^*}\Delta_k$ , with equality at $A = A^*$ ; and since the positive and negative parts of $(\Delta_k)$ have equal total mass, $\sum_{A^*}\Delta_k = \frac12\sum_k\abs{\Delta_k}$ . Exchanging $\mu, \nu$ handles the sign: $d_{\mathrm{TV}}(\mu, \nu) = \frac12\sum_k\abs{\Delta_k}$ . Coupling: for any $A$ ,

\mu(A) - \nu(A) = \E\bigl[\mathbf 1_A(X) - \mathbf 1_A(Y)\bigr] = \E\bigl[(\mathbf 1_A(X) - \mathbf 1_A(Y))\,\mathbf 1_{X\neq Y}\bigr] \leq \P(X \neq Y),

and take the supremum over $A$ .

21. The two laws charge: $k = 0$ : $1 - p$ versus $\eu^{-p}$ , with $\eu^{-p} > 1 - p$ ; $k = 1$ : $p$ versus $p\,\eu^{-p} < p$ ; $k \geq 2$ : $0$ versus the Poisson remainder $1 - \eu^{-p} - p\eu^{-p} \geq 0$ . Hence

d_{\mathrm{TV}} = \tfrac12\bigl[(\eu^{-p} - 1 + p) + (p - p\eu^{-p}) + (1 - \eu^{-p} - p\eu^{-p})\bigr] = \tfrac12\bigl(2p - 2p\eu^{-p}\bigr) = p(1 - \eu^{-p}),

and $1 - \eu^{-p} \leq p$ gives the bound $p^2$ .

22. Write $H_{i-1} = W_i + X_i$ and $H_i = W_i + Y_i$ with $W_i = \sum_{j<i}Y_j + \sum_{j>i}X_j$ , independent of the pair $(X_i, Y_i)$ (coalitions). For $A \subseteq \N$ , conditioning on the countably many values by independence,

\P(H_{i-1} \in A) = \sum_{k\geq0}\P(X_i = k)\,\P(W_i + k \in A),

and likewise for $H_i$ with $Y_i$ . Subtracting, with $c_k = \P(W_i + k \in A) \in \intcc01$ and $\Delta_k = \P(X_i = k) - \P(Y_i = k)$ of zero sum:

\abs{\P(H_{i-1} \in A) - \P(H_i \in A)} = \Bigl|\sum_k\Delta_k\bigl(c_k - \tfrac12\bigr)\Bigr| \leq \tfrac12\sum_k\abs{\Delta_k} = d_{\mathrm{TV}}\bigl(\mathcal B(1, p_i), \mathcal P(p_i)\bigr) .

Telescoping from $H_0 = S$ to $H_n = \sum_iY_i \sim \mathcal P(\lambda)$ (Exercise 23.1, iterated) and using question 21:

\abs{\P(S \in A) - \P(\mathcal P(\lambda) \in A)} \leq \sum_{i=1}^np_i\bigl(1 - \eu^{-p_i}\bigr) \leq \sum_{i=1}^np_i^2 \qquad\text{for every } A :

Le Cam’s inequality. (Question 20’s coupling bound gives an alternative route: couple each pair on one uniform variable so that $\P(X_i \neq Y_i) \leq p_i^2$ and bound $\P(S \neq \sum Y_i)$ ; swapping needs no construction at all.)

23. (a) With $p_i = \frac\lambda n$ : $d_{\mathrm{TV}}(\text{law of }S, \mathcal P(\lambda)) \leq \frac{\lambda^2}n$ . This sharpens Exercise 23.5 three times over: an explicit error at every finite $n$ , uniformity over all events $A$ at once (not one interval at a time), and no need for equal $p_i$ — only $\sum_ip_i^2$ small, e.g. $\sum p_i^2 \leq \lambda\max_ip_i$ : many rare events, none dominant. (b) Here $n = 500$ , $p_i = \frac1{500}$ , $\lambda = 1$ : the Poisson model errs by at most $500\cdot\frac1{500^2} = 0.002$ on every event; in particular, taking $A = \{0\}$ ,

\P(\text{no letter astray}) = \Bigl(1 - \frac1{500}\Bigr)^{500}, \qquad \Bigl|\P(\text{no letter astray}) - \eu^{-1}\Bigr| \leq 0.002,

so the answer is $\eu^{-1} \approx 0.368$ up to a guaranteed $0.002$ (the true discrepancy is about $4\cdot10^{-4}$ ). (c) The problem closes on one method with two regimes. When $n$ comparable contributions each carry variance $\frac1n$ , matching two moments against the Gaussian makes the swap errors $o(\frac1n)$ each: sums go Gaussian — with Taylor as the local comparison tool. When $n$ contributions are indicators of probability $p_i$ , matching the mean against a Poisson atom makes each swap cost $p_i^2$ : counts of rare events go Poisson — with total variation as the exact local comparison. Same hybrids, same telescope, different local estimate: replacement is a strategy, not a theorem, and the Gaussian and Poisson limits are its two oldest dividends.

24. The CLT gives $\sqrt n(\bar X_n - \mu) \Rightarrow \mathcal N(0, \sigma^2)$ , and $g(x) = \ln x$ is differentiable at $\mu > 0$ with $g'(\mu) = \frac1\mu$ : the delta method (Part VI) yields $\sqrt n(\ln\bar X_n - \ln\mu) \Rightarrow \mathcal N(0, \sigma^2/\mu^2)$ . Unwinding the interval $\abs{\ln\bar X_n - \ln\mu} \leq \frac{1.96\,\sigma}{\mu\sqrt n}$ by exponentiation:

\mu \in \bar X_n\cdot \eu^{\pm1.96\,\sigma/(\mu\sqrt n)} \qquad\text{with asymptotic probability } 95\%

(in practice $\sigma/\mu$ is replaced by its empirical version, Slutsky as in Exercise 23.8). The multiplicative interval is the natural one when the data are positive with errors proportional to their size — incomes, concentrations, half-lives: quantities that live on a log scale, where symmetric additive intervals could even cross zero.

25. $\E[(X - p)^3] = (1-p)^3p + (-p)^3(1 - p) = p(1-p)\bigl[(1-p)^2 - p^2\bigr] = p(1-p)(1 - 2p)$ . In the swapping analysis (Part IV), the leading error term after matching two moments carries the signed third moment: for $p < \frac12$ it is positive (the law leans right: rare large excursions above the mean), and the normal approximation systematically misplaces mass — underestimating the short left tail and overestimating the right — with an error of order $n^{-1/2}$ ; at $p = \frac12$ the third moment vanishes, the Bernoulli matches the Gaussian to third order, and the rate improves (Part IV’s matched-moment question). Numerically: $\P(S = 0) = 0.9^{20} = 0.1216$ , while the Gaussian $\mathcal N(2, 1.8)$ gives $\Phi\bigl(\frac{0.5 - 2}{\sqrt{1.8}}\bigr) = \Phi(-1.118) \approx 0.132$ : the normal curve, ignorant of the wall at $0$ and of the rightward skew, puts too much mass at the bottom — the predicted sign of the error, visible at $n = 20$ .