---
title: "Characteristic Functions and the Central Limit Theorem"
book: "University Mathematics — Year 3"
subject: math
language: en
chapter: 23
exercises: 12
source: https://one-course.com/books/math/5/en/chapter/23-characteristic-functions-and-the-central-limit-theorem
---

# Chapter 23 — Characteristic Functions and the Central Limit Theorem

The law of large numbers says averages converge; the central limit theorem says *how they fluctuate*: the error, magnified by $\sqrt n$, is asymptotically [Gaussian](#def-b3-clt-gaussianvector) — whatever the law one started from. This universality is the deepest fact of elementary probability, and its natural proof is Fourier-analytic: the *[characteristic function](#def-b3-clt-cf)* (the Fourier transform of a law) converts [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) sums into products, and [Chapter 14](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#ch-b3-fouriertransform)’s machinery — injectivity, [Gaussian](#def-b3-clt-gaussianvector) fixed points — converts pointwise convergence of these products into convergence of laws (Lévy’s theorem, proved in full). The chapter ends with [Gaussian vectors](#def-b3-clt-gaussianvector) and the honest derivation of the confidence intervals used everywhere in statistics; the weekend problem gives Lindeberg’s second proof of the CLT, with an explicit error rate.

## 23.1 Characteristic functions

**Definition 23.1.**

The *characteristic function* of a real [random variable](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) $X$ is

$$
\varphi_X(\xi) = \E\bigl[\eu^{\iu\xi X}\bigr]
= \int_\R \eu^{\iu\xi x}\,\dd\P_X(x)
\qquad (\xi \in \R)
$$

(the transfer theorem computes it from the law; for a [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) $f$, $\varphi_X(\xi) = \hat f(-\xi)$ in [Chapter 14](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#ch-b3-fouriertransform)’s convention).

**Proposition 23.2.**

(a) $\varphi_X(0) = 1$, $\abs{\varphi_X} \leq 1$, and $\varphi_X$ is uniformly [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity); $\varphi_{aX + b}(\xi) = \eu^{\iu b\xi}\varphi_X(a\xi)$. (b) If $X, Y$ are *[independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence)*: $\varphi_{X+Y} = \varphi_X\,\varphi_Y$. (c) If $\E\abs X^k < \infty$, then $\varphi_X \in \mathcal
C^k$ with $\varphi_X^{(j)}(0) = \iu^j\,\E[X^j]$ for $j \leq
k$; in particular, for centered $X \in L^2$ with variance $\sigma^2$:

$$
\varphi_X(\xi) = 1 - \frac{\sigma^2\xi^2}{2} +
o(\xi^2) \qquad (\xi \to 0).
$$

(d) [Gaussian](#def-b3-clt-gaussianvector): $X \sim \mathcal N(m, \sigma^2)$ has $\varphi_X(\xi) = \eu^{\iu m\xi - \sigma^2\xi^2/2}$.

**Proof.** (a) Bounds are immediate; [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity): $\abs{\varphi(\xi + h)
- \varphi(\xi)} \leq \E\abs{\eu^{\iu hX} - 1} \to 0$ as $h
\to 0$ by dominated convergence, uniformly in $\xi$. The affine rule is a substitution. (b) $\eu^{\iu\xi(X+Y)} =
\eu^{\iu\xi X}\eu^{\iu\xi Y}$, and [expectations](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) of products of [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) variables factor ([Theorem 22.5](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-independence), applied to real and imaginary parts). (c) Differentiation under the [expectation](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space), dominated by $\E\abs X^j$ ([Theorem 10.15](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#thm-b3-lebesgue-paramdiff)); the Taylor expansion at $0$ is then Taylor–Young for the $\mathcal C^2$ function $\varphi$. (d) For $\mathcal N(0,1)$: the [Gaussian](#def-b3-clt-gaussianvector) transform ([Example 14.2](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#ex-b3-fouriertransform-gaussian) with $a = \frac12$) gives $\int\eu^{\iu\xi x}\frac{\eu^{-x^2/2}}{\sqrt{2\pi}}\dd
x = \eu^{-\xi^2/2}$; the general case by the affine rule. ∎

**Theorem 23.3 (Injectivity).**

If $\varphi_X = \varphi_Y$, then $X$ and $Y$ have the same law. More precisely, for $N \sim \mathcal N(0,1)$ [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of $X$ and $\varepsilon > 0$, the smoothed variable $X +
\varepsilon N$ has the [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma)

$$
p_\varepsilon(x) = \frac1{2\pi}\int_\R
\varphi_X(-\xi)\,\eu^{-\varepsilon^2\xi^2/2}\,
\eu^{\iu\xi x}\,\dd\xi ,
$$

determined by $\varphi_X$ alone; letting $\varepsilon \to 0$ recovers the law of $X$.

**Proof.** $X + \varepsilon N$ has the [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) $p_\varepsilon(x) =
\E\bigl[g_\varepsilon(x - X)\bigr]$, where $g_\varepsilon$ is the $\mathcal N(0, \varepsilon^2)$ [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma): indeed for Borel $B$, [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) and Tonelli give $\P(X + \varepsilon N \in
B) = \int\!\!\int\mathbf 1_B(x + \varepsilon
n)g_1(n)\,\dd n\,\dd\P_X(x) = \int_B\E[g_\varepsilon(t -
X)]\dd t$ (substitute, then Tonelli again). Writing $g_\varepsilon$ by Fourier inversion of its transform ([Exercise 14.4](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#exo-b3-fouriertransform-4), rescaled): $g_\varepsilon(u) = \frac1{2\pi}\int
\eu^{-\varepsilon^2\xi^2/2}\eu^{\iu\xi u}\dd\xi$, and Fubini (everything dominated by the [Gaussian](#def-b3-clt-gaussianvector) factor):

$$
p_\varepsilon(x) = \frac1{2\pi}\int_\R
\varphi_X(-\xi)\,\eu^{-\varepsilon^2\xi^2/2}\,
\eu^{\iu\xi x}\,\dd\xi ,
$$

a functional of $\varphi_X$ alone. If $\varphi_X =
\varphi_Y$: $X + \varepsilon N$ and $Y + \varepsilon N$ have equal laws for every $\varepsilon$; for bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f$, $\E f(X + \varepsilon N) \to \E f(X)$ as $\varepsilon
\to 0$ (dominated convergence, $X + \varepsilon N \to X$ pointwise on the product space), so $\E f(X) = \E f(Y)$ for all such $f$ — and this determines the law: for each $t$, squeeze $\mathbf 1_{\intoc{-\infty}t}$ between the bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) ramps $f_k^\pm$ (equal to $1$ on $\intoc{-\infty}{t \mp \frac1k}$, to $0$ beyond $t \pm
\frac1k$, affine between); passing to the limit in $\E
f_k^-(X) \leq F_X(t) \leq \E f_k^+(X)$ gives $F_X(t) =
F_Y(t)$ at every $t$ where both are [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity), hence everywhere by [right-continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) and [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) of common [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points (both $F$’s have countably many jumps); equal distribution functions force equal laws ([Exercise 9.3](https://one-course.com/books/math/5/en/chapter/9-measure-theory#exo-b3-measure-3), resting on [Theorem 9.7](https://one-course.com/books/math/5/en/chapter/9-measure-theory#thm-b3-measure-uniqueness)). ∎

## 23.2 Convergence in distribution

**Definition 23.4.**

$X_n$ *converges in distribution* (or in law) to $X$, written $X_n \Rightarrow X$, if

$$
\E\bigl[f(X_n)\bigr] \longrightarrow \E\bigl[f(X)\bigr]
\qquad\text{for every bounded continuous } f\colon\R\to\R .
$$

Equivalently ([Exercise 23.4](#exo-b3-clt-4)): $F_{X_n}(t) \to F_X(t)$ at every [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) point $t$ of $F_X$. The $X_n$ need not live on a common [probability space](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space): only laws matter.

**Theorem 23.5 (Helly’s selection theorem).**

Every sequence $(F_n)$ of distribution functions has a subsequence converging pointwise, at every [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) point of the limit, to a nondecreasing [right-continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $G \colon
\R \to \intcc01$ — possibly with $G(+\infty) - G(-\infty) <
1$ (mass may escape to infinity).

**Proof.** Diagonal extraction gives $F_{n_k}(q) \to \ell(q)$ for every rational $q$ (values in the [compact](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-compact) $\intcc01$). Define $G(t) = \inf\{\ell(q) : q \in \Q, q > t\}$: nondecreasing; [right-continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) (an infimum over shrinking rational [neighborhoods](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-topology) from the right). At a [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) point $t$ of $G$: for rationals $q_1 < t < q_2$,

$$
\ell(q_1) \leq \liminf F_{n_k}(t) \leq \limsup F_{n_k}(t)
\leq \ell(q_2),
$$

by monotonicity of each $F_{n_k}$. From the definition of $G$ as an infimum and monotonicity of $\ell$ on the rationals: $G(s) \leq \ell(q) \leq G(q)$ whenever $s < q$. Taking $s < q_1 < t$ gives $\ell(q_1) \geq G(s)$, and $\ell(q_2) \leq G(q_2)$; letting $s \uparrow t$ and $q_2
\downarrow t$, [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) of $G$ at $t$ squeezes both the $\liminf$ and the $\limsup$ to $G(t)$. ∎

**Lemma 23.6 (Tightness from the characteristic function).**

For any [random variable](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) $X$ and $u > 0$:

$$
\P\Bigl(\abs X \geq \frac2u\Bigr) \;\leq\;
\frac1u\int_{-u}^{u}\bigl(1 -
\operatorname{Re}\varphi_X(\xi)\bigr)\,\dd\xi .
$$

**Proof.** By Tonelli–Fubini (integrand bounded, region finite in $\xi$):

$$
\frac1u\int_{-u}^u\bigl(1 -
\operatorname{Re}\varphi_X(\xi)\bigr)\dd\xi
= \E\Bigl[\frac1u\int_{-u}^u(1 - \cos(\xi X))\,\dd\xi\Bigr]
= 2\,\E\Bigl[1 - \frac{\sin(uX)}{uX}\Bigr]
$$

(interpret the bracket as its limit $0$ at $X = 0$). The integrand is nonnegative ($\abs{\sin t} \leq \abs t$), and for $\abs{uX} \geq 2$: $1 - \frac{\sin(uX)}{uX} \geq 1 -
\frac1{\abs{uX}} \geq \frac12$. Keeping only the event $\{\abs{uX} \geq 2\}$ inside the [expectation](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) therefore leaves at least $2 \cdot \frac12\,\P(\abs X \geq \frac2u)$, which is the claim. ∎

**Theorem 23.7 (Lévy’s continuity theorem).**

Let $(X_n)$ be [random variables](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) whose [characteristic functions](#def-b3-clt-cf) converge pointwise: $\varphi_{X_n}(\xi) \to
\varphi(\xi)$ for every $\xi$, where $\varphi =
\varphi_X$ is the [characteristic function](#def-b3-clt-cf) of some [random variable](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) $X$. Then $X_n \Rightarrow X$.

**Proof.** *Tightness.* Fix $\varepsilon > 0$. Since $\varphi$ is [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) at $0$ with $\varphi(0) = 1$, choose $u > 0$ with $\frac1u\int_{-u}^u(1 - \operatorname{Re}\varphi) <
\varepsilon$; by dominated convergence (integrand bounded by $2$ on the fixed $[-u,u]$), the same integral for $\varphi_{X_n}$ is $< 2\varepsilon$ for $n$ large: [Lemma 23.6](#lem-b3-clt-tightness) gives $\P(\abs{X_n} \geq \frac2u)
\leq 2\varepsilon$ for large $n$, and enlarging the constant handles the finitely many others: the laws are *tight* — no mass escapes.

*Subsequences.* Let $(F_{n_k})$ be any subsequence; by Helly ([Theorem 23.5](#thm-b3-clt-helly)) extract $F_{n_{k_j}} \to G$ at [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points. Tightness forces $G(-\infty) = 0$, $G(+\infty) = 1$ ($G(\frac2u) - G(-\frac2u) \geq 1 -
2\varepsilon$ at [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points): $G$ is a genuine distribution function, of some [random variable](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) $Y$. Then $X_{n_{k_j}} \Rightarrow Y$ ([Exercise 23.4](#exo-b3-clt-4), distributional convergence from $F$’s), so $\varphi_{X_{n_{k_j}}} \to \varphi_Y$ *pointwise* ($x
\mapsto \eu^{\iu\xi x}$ is bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity), real and imaginary parts separately); comparing with the hypothesis: $\varphi_Y = \varphi = \varphi_X$, and injectivity ([Theorem 23.3](#thm-b3-clt-injectivity)) gives $Y \sim X$, i.e. $G =
F_X$.

*Conclusion.* Every subsequence of $(F_n)$ has a sub-subsequence converging to the *same* $F_X$ (at its [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points); hence $F_n(t) \to F_X(t)$ at every [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) point $t$ (a real sequence all of whose subsequences have subsubsequences with the same limit converges): $X_n \Rightarrow X$. ∎

## 23.3 The central limit theorem

**Theorem 23.8 (Central limit theorem).**

Let $(X_n)$ be i.i.d. with $\E X_1 = m$ and $\V(X_1) =
\sigma^2 \in \intoo0\infty$. Then

$$
\frac{S_n - nm}{\sigma\sqrt n} \;\Longrightarrow\; \mathcal
N(0, 1) :
\qquad
\P\Bigl(a \leq \frac{S_n - nm}{\sigma\sqrt n} \leq
b\Bigr) \longrightarrow
\frac{1}{\sqrt{2\pi}}\int_a^b\eu^{-x^2/2}\,\dd x
$$

for all $a < b$.

**Proof.** [Center](https://one-course.com/books/math/5/en/chapter/1-group-theory#ex-b3-groups-actions) and normalize: $Z_i = \frac{X_i - m}{\sigma}$ (i.i.d., mean $0$, variance $1$) and $T_n =
\frac1{\sqrt n}\sum_{i\leq n}Z_i$. By [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) and the affine rule ([Proposition 23.2](#prop-b3-clt-cfbasics)):

$$
\varphi_{T_n}(\xi) =
\varphi_{Z}\Bigl(\frac{\xi}{\sqrt n}\Bigr)^{n},
\qquad
\varphi_Z(\eta) = 1 - \frac{\eta^2}2 + \eta^2\rho(\eta),\quad
\rho(\eta)\to0 .
$$

Fix $\xi$ and let $a_n = \varphi_Z(\xi/\sqrt n)$, $b_n = 1 -
\frac{\xi^2}{2n}$: both have modulus $\leq 1$ for $n$ large ($\abs{b_n} \leq 1$ once $\xi^2 \leq 4n$; $\abs{a_n} \leq 1$ always). The elementary inequality $\abs{a^n - b^n} \leq
n\abs{a - b}$ for $\abs a, \abs b \leq 1$ (telescoping $a^n - b^n = \sum a^k(a - b)b^{n-1-k}$) gives

$$
\Bigl|\varphi_{T_n}(\xi) - \Bigl(1 -
\frac{\xi^2}{2n}\Bigr)^{n}\Bigr|
\leq n\,\Bigl|\varphi_Z\Bigl(\frac\xi{\sqrt n}\Bigr) - 1 +
\frac{\xi^2}{2n}\Bigr|
= \xi^2\,\Bigl|\rho\Bigl(\frac{\xi}{\sqrt n}\Bigr)\Bigr|
\longrightarrow 0,
$$

while $\bigl(1 - \frac{\xi^2}{2n}\bigr)^n \to
\eu^{-\xi^2/2}$ (real logarithm). So $\varphi_{T_n}(\xi) \to
\eu^{-\xi^2/2} = \varphi_{\mathcal N(0,1)}(\xi)$ ([Proposition 23.2](#prop-b3-clt-cfbasics)(d)) for every $\xi$: Lévy ([Theorem 23.7](#thm-b3-clt-levy)) concludes $T_n \Rightarrow \mathcal
N(0,1)$. The interval probabilities follow since $F_{\mathcal
N}$ is [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) everywhere. ∎

**Example 23.9 (Confidence intervals, honestly derived).**

Poll $n$ [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) voters; $\hat p_n = S_n/n$ estimates the true $p$, with $\sigma^2 = p(1-p) \leq \frac14$. The CLT gives, for large $n$,

$$
\P\Bigl(\abs{\hat p_n - p} \leq
\frac{z}{2\sqrt n}\Bigr)
\;\geq\; \P\Bigl(\Bigl|\frac{S_n - np}{\sigma\sqrt n}\Bigr|
\leq z\Bigr)
\longrightarrow \Phi(z) - \Phi(-z),
$$

where $\Phi$ is the standard [Gaussian](#def-b3-clt-gaussianvector) distribution function. With $z = 1.96$: asymptotic confidence $95\%$, and margin $\frac{1.96}{2\sqrt n} \leq 3\%$ requires $n \geq
\bigl(\frac{1.96}{0.06}\bigr)^2 \approx 1068$ — the number behind every “$\pm3$ points, $95\%$” one reads; compare Chebyshev’s $5556$ ([Exercise 22.7](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#exo-b3-probability-7)). The $\sqrt n$ is universal: to halve the error, quadruple the sample — the same law that fixes [Monte Carlo](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#ex-b3-probability-sllnapps)’s cost ([Exercise 23.7](#exo-b3-clt-7)).

## 23.4 Gaussian vectors

**Definition 23.10.**

A random vector $X = (X_1, \dots, X_d)$ is *Gaussian* if every linear combination $\langle t, X\rangle = \sum t_iX_i$ is a (possibly degenerate) real Gaussian variable. Its law is determined by the mean vector $m = (\E X_i)$ and the *covariance matrix* $\Sigma = \bigl(\operatorname{Cov}
(X_i, X_j)\bigr)$: indeed the [characteristic function](#def-b3-clt-cf) of the vector, $\varphi_X(t) = \E\eu^{\iu\langle t, X\rangle}$, is the value at $1$ of the cf of $\langle t, X\rangle$:

$$
\varphi_X(t) = \exp\Bigl(\iu\langle t, m\rangle -
\tfrac12\,t^{\mathsf T}\Sigma\,t\Bigr),
$$

and $d$-dimensional [characteristic functions](#def-b3-clt-cf) are injective (same smoothing proof as [Theorem 23.3](#thm-b3-clt-injectivity), coordinatewise Gaussians).

**Theorem 23.11.**

Let $X$ be a [Gaussian vector](#def-b3-clt-gaussianvector).

1. Every affine image $AX + b$ is a [Gaussian vector](#def-b3-clt-gaussianvector) .
2. The components $X_i$ are *[independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence)* if and only if $\Sigma$ is diagonal: for jointly [Gaussian](#def-b3-clt-gaussianvector) variables, uncorrelated $=$ [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) .
3. If $\Sigma$ is invertible, $X$ has the [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) $\frac{1}{(2\pi)^{d/2}\sqrt{\det\Sigma}}  \exp\bigl(-\frac12(x - m)^{\mathsf T}\Sigma^{-1}(x -  m)\bigr)$ .

**Proof.** (1) Linear combinations of components of $AX + b$ are affine functions of linear combinations of $X$: [Gaussian](#def-b3-clt-gaussianvector) (an affine image of a [Gaussian](#def-b3-clt-gaussianvector) variable is [Gaussian](#def-b3-clt-gaussianvector)). (2) If $\Sigma$ is diagonal, the [characteristic function](#def-b3-clt-cf) factorizes: $\varphi_X(t) = \prod_i\exp(\iu t_im_i -
\frac12\Sigma_{ii}t_i^2) = \prod\varphi_{X_i}(t_i)$, which is the [characteristic function](#def-b3-clt-cf) of the product law ([Theorem 22.5](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-independence) read through $d$-dimensional injectivity): the components are [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence). The converse is the vanishing of covariances of [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) $L^2$ variables. (3) Diagonalize $\Sigma = P D P^{\mathsf T}$ ($P$ orthogonal, $D > 0$ diagonal — [Exercise 20.8](https://one-course.com/books/math/5/en/chapter/20-submanifolds-of-rn#exo-b3-submanifolds-8)); the vector $Y = P^{\mathsf T}(X - m)$ is [Gaussian](#def-b3-clt-gaussianvector) with covariance $D$: by (2) its components are [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) $\mathcal N(0, d_i)$, so $Y$ has the product [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma); push forward by the volume-preserving $x = m + PY$ ([Theorem 11.10](https://one-course.com/books/math/5/en/chapter/11-product-measures-fubini-change-of-variables#thm-b3-product-linearchange), $\abs{\det P} = 1$) and rewrite the exponent invariantly. ∎

**Theorem 23.12 (Multidimensional CLT).**

Let $(X_n)$ be i.i.d. [square-integrable](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-l1) random *vectors* of $\R^d$ with mean $m$ and covariance matrix $\Sigma$. Then $\frac{S_n - nm}{\sqrt n}$ converges in distribution to the [Gaussian vector](#def-b3-clt-gaussianvector) $\mathcal N(0,
\Sigma)$.

**Proof.** *Admitted at this level.* ∎

**Remark 23.13.**

Almost everything is already in our hands. For each direction $t \in \R^d$, the real variable $\langle t,
\frac{S_n - nm}{\sqrt n}\rangle$ is a normalized sum of i.i.d. real variables of variance $t^{\mathsf T}\Sigma t$, so [Theorem 23.8](#thm-b3-clt-clt)’s computation gives pointwise convergence of the $d$-dimensional [characteristic functions](#def-b3-clt-cf) to $\eu^{-t^{\mathsf T}\Sigma t/2}$, the [characteristic function](#def-b3-clt-cf) of $\mathcal N(0, \Sigma)$ ([Definition 23.10](#def-b3-clt-gaussianvector)). What we have not re-proved is Lévy’s [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) theorem *in $\R^d$*: Helly’s selection and the tightness estimate generalize routinely (coordinatewise), and this Cramér–Wold reduction is carried out honestly in any graduate probability course; nothing beyond this chapter’s methods is needed.

**Method 23.14.**

To identify a limit law: compute [characteristic functions](#def-b3-clt-cf), take the pointwise limit, recognize it ([Gaussian](#def-b3-clt-gaussianvector) $\eu^{-\sigma^2\xi^2/2}$, Poisson $\eu^{\lambda(\eu^{\iu\xi}-1)}$, exponential $\frac{\lambda}
{\lambda - \iu\xi}$, …) and invoke Lévy. The three-step ritual ([independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) $\to$ product; Taylor at $0$ $\to$ exponential limit; Lévy $\to$ convergence in law) proves the CLT, the Poisson law of rare events ([Exercise 23.5](#exo-b3-clt-5)), and every classical limit theorem of this course. For a.s. statements, go back to [Chapter 22](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#ch-b3-probability)’s toolkit: the two chapters answer different questions about the same $S_n$.

## 23.5 Exercises

**Exercise 23.1 ★.**

Compute the [characteristic functions](#def-b3-clt-cf): uniform on $\intcc{-1}1$; exponential $\mathcal E(\lambda)$; Poisson $\mathcal P(\lambda)$; binomial $\mathcal B(n, p)$. Deduce via [Theorem 23.3](#thm-b3-clt-injectivity) that the sum of [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) Poisson variables ($\lambda, \mu$) is Poisson $(\lambda +
\mu)$.

**Solution of Exercise 23.1.**

Uniform on $\intcc{-1}1$: $\varphi(\xi) =
\frac12\int_{-1}^1\eu^{\iu\xi x}\dd x =
\frac{\sin\xi}{\xi}$ (equal to $1$ at $\xi = 0$). Exponential $\mathcal E(\lambda)$: $\varphi(\xi) =
\lambda\int_0^\infty\eu^{(\iu\xi - \lambda)x}\dd x =
\frac{\lambda}{\lambda - \iu\xi}$ (the antiderivative vanishes at $+\infty$ since $\operatorname{Re}(\iu\xi -
\lambda) < 0$). Poisson $\mathcal P(\lambda)$: by the transfer theorem for discrete laws,

$$
\varphi(\xi) = \sum_{k\geq0}\eu^{\iu\xi
k}\,\eu^{-\lambda}\frac{\lambda^k}{k!}
= \eu^{-\lambda}\exp\bigl(\lambda\eu^{\iu\xi}\bigr)
= \exp\bigl(\lambda(\eu^{\iu\xi} - 1)\bigr).
$$

Binomial $\mathcal B(n, p)$: a sum of $n$ [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) Bernoulli variables, each with cf $1 - p + p\eu^{\iu\xi}$, so $\varphi(\xi) = \bigl(1 - p + p\eu^{\iu\xi}\bigr)^n$ ([Proposition 23.2](#prop-b3-clt-cfbasics)(b)). Poisson additivity: if $X
\sim \mathcal P(\lambda)$, $Y \sim \mathcal P(\mu)$ are [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence),

$$
\varphi_{X+Y}(\xi) = \eu^{\lambda(\eu^{\iu\xi}-1)}
\eu^{\mu(\eu^{\iu\xi}-1)} =
\eu^{(\lambda+\mu)(\eu^{\iu\xi}-1)},
$$

the cf of $\mathcal P(\lambda + \mu)$; injectivity ([Theorem 23.3](#thm-b3-clt-injectivity)) identifies the law.

**Exercise 23.2 ★★.**

(a) Show that $\varphi_X$ is real-valued if and only if $X$ and $-X$ have the same law (a *symmetric* variable). (b) Suppose $\abs{\varphi_X(\xi_0)} = 1$ for some $\xi_0 \neq
0$. Show that $X$ is almost surely supported on an arithmetic progression $a + \frac{2\pi}{\xi_0}\Z$ *(write $\varphi_X(\xi_0) = \eu^{\iu\theta}$ and compute $\E[1 -
\cos(\xi_0X - \theta)]$)*. Deduce that if $X$ has a [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma), then $\abs{\varphi_X(\xi)} < 1$ for all $\xi \neq 0$.

**Solution of Exercise 23.2.**

(a) $\overline{\varphi_X(\xi)} = \E\eu^{-\iu\xi X} =
\varphi_{-X}(\xi)$. So $\varphi_X$ is real if and only if $\varphi_X = \varphi_{-X}$, if and only if (injectivity, [Theorem 23.3](#thm-b3-clt-injectivity)) $X$ and $-X$ have the same law. (b) Write $\varphi_X(\xi_0) = \eu^{\iu\theta}$. Then

$$
\E\bigl[1 - \cos(\xi_0X - \theta)\bigr]
= 1 - \operatorname{Re}\bigl(\eu^{-\iu\theta}
\varphi_X(\xi_0)\bigr) = 1 - 1 = 0 .
$$

The integrand is nonnegative, so $\cos(\xi_0X - \theta) = 1$ almost surely (a nonnegative variable with zero [expectation](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) vanishes a.s.), i.e. $\xi_0X - \theta \in 2\pi\Z$ a.s.: $X$ takes its values in the arithmetic progression $\frac{\theta}{\xi_0} + \frac{2\pi}{\xi_0}\Z$ almost surely. If $X$ has a [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma), this countable set is Lebesgue-null, so it carries probability $0$ — contradiction; therefore $\abs{\varphi_X(\xi)} < 1$ for every $\xi \neq 0$.

**Exercise 23.3 ★★.**

Let $X \sim \mathcal N(m_1, \sigma_1^2)$ and $Y \sim
\mathcal N(m_2, \sigma_2^2)$ be [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence). Show $X + Y
\sim \mathcal N(m_1 + m_2, \sigma_1^2 + \sigma_2^2)$, and more generally that the [Gaussian](#def-b3-clt-gaussianvector) family is stable under [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) sums and affine maps. Contrast: is the sum of two *dependent* [Gaussians](#def-b3-clt-gaussianvector) always [Gaussian](#def-b3-clt-gaussianvector)? ([Exercise 23.9](#exo-b3-clt-9).)

**Solution of Exercise 23.3.**

By [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) and [Proposition 23.2](#prop-b3-clt-cfbasics):

$$
\varphi_{X+Y}(\xi) = \eu^{\iu m_1\xi - \sigma_1^2\xi^2/2}\,
\eu^{\iu m_2\xi - \sigma_2^2\xi^2/2}
= \eu^{\iu(m_1+m_2)\xi - (\sigma_1^2+\sigma_2^2)\xi^2/2},
$$

the cf of $\mathcal N(m_1 + m_2, \sigma_1^2 + \sigma_2^2)$; injectivity concludes. Stability under affine maps is the affine rule ($aX + b \sim \mathcal N(am_1 + b,
a^2\sigma_1^2)$, allowing the degenerate case $a = 0$), and stability under [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) sums follows by induction on the computation above. For *dependent* [Gaussians](#def-b3-clt-gaussianvector) the sum need not be [Gaussian](#def-b3-clt-gaussianvector): in [Exercise 23.9](#exo-b3-clt-9), $X$ and $Y =
\varepsilon X$ are each standard [Gaussian](#def-b3-clt-gaussianvector) but $X + Y$ vanishes with probability $\frac12$ without being a.s. zero, so it is not [Gaussian](#def-b3-clt-gaussianvector).

**Exercise 23.4 ★★.**

(a) Prove the equivalence in [Definition 23.4](#def-b3-clt-cid): if $\E f(X_n) \to \E f(X)$ for all bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f$, then $F_{X_n}(t) \to F_X(t)$ at [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points *(squeeze $\mathbf 1_{\intoc{-\infty}t}$ between two [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) staircase-ramps)*; and conversely *(approximate a bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f$ by sums of ramp functions, or condition on a fine grid of [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points)* — the converse may be treated for $f$ uniformly [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) first, then in general. (b) Show that $X_n \Rightarrow c$ (a constant) implies $X_n
\to c$ in probability.

**Solution of Exercise 23.4.**

(a) *Direct implication.* Let $t$ be a [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) point of $F_X$ and $\delta > 0$. Take the [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) ramps $f^-$ ($= 1$ on $\intoc{-\infty}{t-\delta}$, $0$ from $t$ on, affine between) and $f^+$ ($= 1$ on $\intoc{-\infty}t$, $0$ from $t + \delta$ on, affine between); then $f^- \leq
\mathbf 1_{\intoc{-\infty}t} \leq f^+$, so

$$
\E f^-(X_n) \leq F_{X_n}(t) \leq \E f^+(X_n),
$$

and the outer terms converge to $\E f^\pm(X)$, themselves squeezed between $F_X(t - \delta)$ and $F_X(t + \delta)$. Letting $n \to \infty$ then $\delta \to 0$ and using [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) of $F_X$ at $t$: $F_{X_n}(t) \to F_X(t)$.

*Converse.* Let $f$ be bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity), $M =
\sup\abs f$, $\varepsilon > 0$. The [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points of $F_X$ are dense ($F_X$ has at most countably many jumps), so choose [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points $a < b$ with $F_X(a) < \varepsilon$ and $1 - F_X(b) < \varepsilon$. On the [compact](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-compact) $\intcc ab$ the function $f$ is uniformly [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity): choose [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points $a = t_0 < t_1 < \dots < t_m = b$ of $F_X$ with oscillation of $f$ at most $\varepsilon$ on each $\intoc{t_{j-1}}{t_j}$, and set $g = \sum_j
f(t_j)\,\mathbf 1_{\intoc{t_{j-1}}{t_j}}$. Then $\abs{f - g}
\leq \varepsilon$ on $\intoc ab$, $\abs g \leq M$, and for $T = X_n$ or $X$:

$$
\bigl|\E f(T) - \E g(T)\bigr| \leq \varepsilon +
2M\bigl(F_T(a) + 1 - F_T(b)\bigr).
$$

Moreover $\E g(X_n) = \sum_j f(t_j)\bigl(F_{X_n}(t_j) -
F_{X_n}(t_{j-1})\bigr) \to \E g(X)$ (a finite sum of converging terms, all $t_j$ being [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points), and $F_{X_n}(a) \to F_X(a) < \varepsilon$, $1 - F_{X_n}(b) \to 1
- F_X(b) < \varepsilon$. Assembling: $\limsup_n\abs{\E f(X_n) - \E f(X)} \leq 2\varepsilon +
8M\varepsilon$; let $\varepsilon \to 0$.

(b) The distribution function of the constant $c$ is $\mathbf 1_{\intco c\infty}$, [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) except at $c$. For $\varepsilon > 0$, the points $c - \varepsilon$ and $c +
\frac\varepsilon2$ are [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points, so

$$
\P(\abs{X_n - c} > \varepsilon) \leq F_{X_n}(c -
\varepsilon) + 1 - F_{X_n}\Bigl(c + \frac\varepsilon2\Bigr)
\longrightarrow 0 + 1 - 1 = 0 .
$$

**Exercise 23.5 ★★.**

(Law of rare events) Let $X_n \sim \mathcal B(n, p_n)$ with $np_n \to \lambda > 0$. Show, via [characteristic functions](#def-b3-clt-cf) and [Theorem 23.7](#thm-b3-clt-levy), that $X_n \Rightarrow \mathcal
P(\lambda)$. Numerical sanity check: compare $\P(X = 0)$ for $\mathcal B(100, 0.02)$ and $\mathcal P(2)$.

**Solution of Exercise 23.5.**

Let $z_n = p_n(\eu^{\iu\xi} - 1)$, so $\varphi_{X_n}(\xi) =
(1 + z_n)^n$ ([Exercise 23.1](#exo-b3-clt-1)) and $\abs{z_n} \leq 2p_n
\to 0$ (note $p_n = \frac{np_n}n \to 0$). Both $1 + z_n$ and $\eu^{z_n}$ have modulus at most $1$: $\abs{1 + z_n} =
\abs{(1 - p_n) + p_n\eu^{\iu\xi}} \leq 1$ by the triangle inequality, and $\abs{\eu^{z_n}} = \eu^{p_n(\cos\xi - 1)}
\leq 1$. The telescoping inequality $\abs{a^n - b^n} \leq
n\abs{a - b}$ (proof of [Theorem 23.8](#thm-b3-clt-clt)) and the power series bound $\abs{\eu^z - 1 - z} \leq
\abs z^2\eu^{\abs z}$ give

$$
\bigl|(1 + z_n)^n - \eu^{nz_n}\bigr| \leq n\bigl|1 + z_n -
\eu^{z_n}\bigr| \leq n\,\abs{z_n}^2\,\eu^{\abs{z_n}} \leq
4\eu^2\,np_n^2 = 4\eu^2\,(np_n)\,p_n \longrightarrow 0 .
$$

Since $nz_n = np_n(\eu^{\iu\xi} - 1) \to
\lambda(\eu^{\iu\xi} - 1)$, we conclude $\varphi_{X_n}(\xi)
\to \exp\bigl(\lambda(\eu^{\iu\xi} - 1)\bigr)$ for every $\xi$: the cf of $\mathcal P(\lambda)$, and Lévy ([Theorem 23.7](#thm-b3-clt-levy)) gives $X_n \Rightarrow \mathcal
P(\lambda)$. Numerically: $\P\bigl(\mathcal B(100, 0.02) =
0\bigr) = 0.98^{100} = \eu^{100\ln 0.98} \approx
\eu^{-2.020} \approx 0.1326$, while $\P\bigl(\mathcal P(2) =
0\bigr) = \eu^{-2} \approx 0.1353$: two percent apart already at this coarse $n$.

**Exercise 23.6 ★★.**

(a) A fair die is rolled $n = 1000$ times; approximate the probability that the total exceeds $3600$ (mean $3500$, variance per roll $\frac{35}{12}$). (b) For $S \sim \mathcal B(100, \frac12)$, approximate $\P(45 \leq S \leq 55)$ by the CLT with the [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) correction ($\pm\frac12$), and comment on the correction’s effect.

**Solution of Exercise 23.6.**

(a) One roll has mean $\frac72$ and variance $\frac{35}{12}$, so $S$ has mean $3500$, variance $\frac{35000}{12} \approx 2916.7$ and standard deviation $\approx 54.0$. By the CLT,

$$
\P(S > 3600) = \P\Bigl(\frac{S - 3500}{54.0} > 1.85\Bigr)
\approx 1 - \Phi(1.85) \approx 0.032 :
$$

about a $3\%$ chance. (b) $S \sim \mathcal B(100, \frac12)$: mean $50$, standard deviation $5$. With the [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) correction,

$$
\P(45 \leq S \leq 55) \approx
\Phi\Bigl(\frac{55.5 - 50}{5}\Bigr) -
\Phi\Bigl(\frac{44.5 - 50}{5}\Bigr) = 2\Phi(1.1) - 1
\approx 0.729,
$$

against the exact value $0.7287$; without the correction, $2\Phi(1) - 1 \approx 0.683$, off by almost five points. The correction matters because $S$ is a lattice variable: the atom $\P(S = k)$ is well approximated by the [Gaussian](#def-b3-clt-gaussianvector) mass of $\intcc{k - \frac12}{k + \frac12}$, and clipping the interval at the integers $45$ and $55$ discards half an atom at each end.

**Exercise 23.7 ★★.**

([Monte Carlo](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#ex-b3-probability-sllnapps) error) In the setting of [Problem 22.1](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#pb-b3-probability-1), question 11, with $g \in
L^2(\intcc01^d)$, let $\sigma^2 = \V(g(U_1))$ and $I = \int
g$. Show

$$
\sqrt n\,\Bigl(\frac1n\sum_{k\leq n}g(U_k) - I\Bigr)
\Longrightarrow \mathcal N(0, \sigma^2),
$$

and deduce the asymptotic $95\%$ error bar $\pm
1.96\,\sigma/\sqrt n$ — [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of the dimension $d$. Compare with the deterministic midpoint rule in dimension $d$ (error $\sim n^{-2/d}$ for $\mathcal C^2$ integrands): from which dimension on does random sampling win?

**Solution of Exercise 23.7.**

The variables $g(U_k)$ are i.i.d. ([measurable](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-measurable) images of i.i.d. variables), [square-integrable](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-l1), with mean $I$ (transfer theorem, [Exercise 11.9](https://one-course.com/books/math/5/en/chapter/11-product-measures-fubini-change-of-variables#exo-b3-product-9)) and variance $\sigma^2$. If $\sigma > 0$, [Theorem 23.8](#thm-b3-clt-clt) applied to them is exactly the stated convergence

$$
\sqrt n\,\Bigl(\frac1n\sum_{k\leq n}g(U_k) - I\Bigr) =
\frac{\sum_{k\leq n}\bigl(g(U_k) - I\bigr)}{\sqrt n}
\Longrightarrow \mathcal N(0, \sigma^2)
$$

(if $\sigma = 0$, $g$ is a.s. constant and the left-hand side vanishes identically). Hence $\P\bigl(\abs{\frac1n\sum g(U_k) - I} \leq
1.96\,\sigma/\sqrt n\bigr) \to 0.95$: the error bar $\pm
1.96\,\sigma/\sqrt n$ sees the dimension $d$ only through the constant $\sigma$, never through the rate in $n$. The midpoint rule with $n$ nodes in dimension $d$ has mesh $n^{-1/d}$ and error of order $n^{-2/d}$ for $\mathcal C^2$ integrands. [Monte Carlo](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#ex-b3-probability-sllnapps)’s $n^{-1/2}$ decays faster than $n^{-2/d}$ exactly when $\frac12 > \frac2d$, i.e. $d > 4$: from dimension $5$ on, random sampling asymptotically beats the grid — the curse of dimensionality spares probabilistic methods, which is why [Monte Carlo](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#ex-b3-probability-sllnapps) rules high-dimensional integration.

**Exercise 23.8 ★★★.**

(Slutsky) Suppose $X_n \Rightarrow X$ and $Y_n \to c$ in probability ($c$ constant). Show $X_n + Y_n \Rightarrow X +
c$ and $Y_nX_n \Rightarrow cX$. *(Work with [characteristic functions](#def-b3-clt-cf) and the bound $\abs{\E\eu^{\iu\xi
(X_n+Y_n)} - \eu^{\iu\xi c}\E\eu^{\iu\xi X_n}} \leq
\E\abs{\eu^{\iu\xi(Y_n - c)} - 1}$, split on $\abs{Y_n - c}
\leq \delta$.)* Application: in [Example 23.9](#ex-b3-clt-confidence), justify replacing the unknown $\sigma = \sqrt{p(1-p)}$ by $\sqrt{\hat p_n(1 - \hat
p_n)}$.

**Solution of Exercise 23.8.**

*Sum.* For fixed $\xi$:

$$
\bigl|\E\eu^{\iu\xi(X_n+Y_n)} -
\eu^{\iu\xi c}\,\E\eu^{\iu\xi X_n}\bigr|
= \bigl|\E\bigl[\eu^{\iu\xi X_n}\bigl(\eu^{\iu\xi Y_n} -
\eu^{\iu\xi c}\bigr)\bigr]\bigr|
\leq \E\bigl|\eu^{\iu\xi(Y_n - c)} - 1\bigr| .
$$

Split on the event $\{\abs{Y_n - c} \leq \delta\}$: there, $\abs{\eu^{\iu\xi(Y_n-c)} - 1} \leq \abs\xi\,\delta$ (the chord is shorter than the arc); the complement contributes at most $2\,\P(\abs{Y_n - c} > \delta) \to 0$. Hence the $\limsup$ is $\leq \abs\xi\,\delta$ for every $\delta > 0$: the difference tends to $0$. Since $\E\eu^{\iu\xi X_n} \to
\varphi_X(\xi)$, we get $\varphi_{X_n+Y_n}(\xi) \to
\eu^{\iu\xi c}\varphi_X(\xi) = \varphi_{X+c}(\xi)$, and Lévy ([Theorem 23.7](#thm-b3-clt-levy)) yields $X_n + Y_n
\Rightarrow X + c$.

*Product.* First, $cX_n \Rightarrow cX$: $\varphi_{cX_n}(\xi) = \varphi_{X_n}(c\xi) \to
\varphi_X(c\xi) = \varphi_{cX}(\xi)$. Next, $(Y_n - c)X_n
\to 0$ in probability: the laws of the $X_n$ are tight (their cfs converge to a cf; see the tightness step of [Theorem 23.7](#thm-b3-clt-levy)), so given $\varepsilon > 0$ pick $M$ with $\P(\abs{X_n} > M) \leq \varepsilon$ for all $n$; then

$$
\P\bigl(\abs{(Y_n - c)X_n} > \varepsilon\bigr) \leq
\P(\abs{X_n} > M) + \P\Bigl(\abs{Y_n - c} >
\frac{\varepsilon}{M}\Bigr) \leq \varepsilon + o(1) .
$$

Writing $Y_nX_n = cX_n + (Y_n - c)X_n$ and applying the sum part (whose proof only used $Y_n' := (Y_n - c)X_n \to 0$ in probability, with constant $0$): $Y_nX_n \Rightarrow cX$.

*Application.* By the strong law of large numbers ([Theorem 22.13](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-slln)), $\hat p_n \to p$ a.s., so by [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $\hat\sigma_n = \sqrt{\hat p_n(1 - \hat p_n)}
\to \sigma = \sqrt{p(1 - p)} > 0$ a.s., hence $\frac{\sigma}{\hat\sigma_n} \to 1$ in probability. Slutsky’s product rule upgrades $\frac{S_n - np}{\sigma\sqrt n}
\Rightarrow \mathcal N(0,1)$ to $\frac{S_n -
np}{\hat\sigma_n\sqrt n} = \frac{\sigma}{\hat\sigma_n}\cdot
\frac{S_n - np}{\sigma\sqrt n} \Rightarrow \mathcal N(0,1)$: the *usable* confidence interval $\hat p_n \pm
1.96\,\hat\sigma_n/\sqrt n$, built from the data alone, keeps its asymptotic $95\%$ level.

**Exercise 23.9 ★★★.**

Let $X \sim \mathcal N(0,1)$ and $\varepsilon$ [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) with $\P(\varepsilon = \pm1) = \frac12$; set $Y =
\varepsilon X$. (a) Show $Y \sim \mathcal N(0,1)$ and $\operatorname{Cov}(X, Y) = 0$. (b) Show $X$ and $Y$ are *not* [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence), and that $(X, Y)$ is not a [Gaussian vector](#def-b3-clt-gaussianvector) *(compute $\P(X + Y =
0)$)*. (c) Moral: [Theorem 23.11](#thm-b3-clt-gaussianvector)(2) requires joint Gaussianity — “uncorrelated [Gaussians](#def-b3-clt-gaussianvector)” alone proves nothing.

**Solution of Exercise 23.9.**

(a) Splitting the [expectation](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) over the two values of $\varepsilon$ ([independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence)): for Borel $B$, $\P(Y \in B) =
\frac12\P(X \in B) + \frac12\P(-X \in B) = \P(X \in B)$, since $-X \sim X$ ($\mathcal N(0,1)$ is symmetric): $Y \sim
\mathcal N(0,1)$. And $\operatorname{Cov}(X, Y) =
\E[\varepsilon X^2] = \E[\varepsilon]\,\E[X^2] = 0 \cdot 1 =
0$. (b) $\abs Y = \abs X$, so $\P(\abs X \leq 1,\ \abs Y \geq 2)
= 0$ while $\P(\abs X \leq 1)\,\P(\abs Y \geq 2) > 0$: not [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence). If $(X, Y)$ were a [Gaussian vector](#def-b3-clt-gaussianvector), $X + Y =
(1 + \varepsilon)X$ would be a real [Gaussian](#def-b3-clt-gaussianvector) variable ([Definition 23.10](#def-b3-clt-gaussianvector) with $t = (1,1)$); but $\P(X + Y = 0) = \P(\varepsilon = -1) = \frac12$, whereas a [Gaussian](#def-b3-clt-gaussianvector) variable has an atom only if it is a.s. constant — and $X + Y$ equals $2X \neq 0$ a.s. on $\{\varepsilon = 1\}$. Contradiction: $(X, Y)$ is not [Gaussian](#def-b3-clt-gaussianvector). (c) Each marginal is [Gaussian](#def-b3-clt-gaussianvector) and the covariance vanishes, yet [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) fails — because the *pair* is not jointly [Gaussian](#def-b3-clt-gaussianvector). [Theorem 23.11](#thm-b3-clt-gaussianvector)(2) cannot be weakened to “[Gaussian](#def-b3-clt-gaussianvector) marginals”.

**Exercise 23.10 ★★.**

The Cauchy law has [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) $\frac1{\pi(1 + x^2)}$. (a) Show its [characteristic function](#def-b3-clt-cf) is $\eu^{-\abs\xi}$ ([Exercise 14.1](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#exo-b3-fouriertransform-1) and inversion). (b) Show that if $X_1, \dots, X_n$ are i.i.d. Cauchy, then $\frac{S_n}n$ is again Cauchy — the *same* law: the average never concentrates. (c) Reconcile with the laws of large numbers and the CLT: which hypotheses fail? (Compute $\E\abs{X_1}$.)

**Solution of Exercise 23.10.**

(a) [Exercise 14.1](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#exo-b3-fouriertransform-1) computes $\widehat{\eu^{-\abs\cdot}}(\xi) = \frac{2}{1 + \xi^2}$; both sides being [integrable](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-l1), Fourier inversion ([Theorem 14.5](https://one-course.com/books/math/5/en/chapter/14-the-fourier-transform#thm-b3-fouriertransform-inversion)) turns this around:

$$
\int_\R\eu^{\iu\xi x}\,\frac{\dd x}{\pi(1 + x^2)} =
\eu^{-\abs\xi},
$$

which is exactly $\varphi_X(\xi)$ for a Cauchy variable $X$. (b) By [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence), $\varphi_{S_n}(\xi) =
\bigl(\eu^{-\abs\xi}\bigr)^n = \eu^{-n\abs\xi}$, so $\varphi_{S_n/n}(\xi) = \varphi_{S_n}(\xi/n) =
\eu^{-\abs\xi}$: the empirical mean $\frac{S_n}n$ is again standard Cauchy for every $n$ (injectivity). The average never concentrates: its fluctuations at time $10^6$ are those of a single observation. (c) $\E\abs{X_1} = \frac2\pi\int_0^\infty\frac{x\,\dd x}{1 +
x^2} = +\infty$: the Cauchy law is not [integrable](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-l1), so the strong law of large numbers ([Theorem 22.13](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-slln)) does not apply, and the CLT (which needs a finite variance) even less. Here their conclusions genuinely fail, not merely their proofs. Consistency check: $\varphi(\xi) = \eu^{-\abs\xi}$ is not differentiable at $0$, as [Proposition 23.2](#prop-b3-clt-cfbasics)(c) read contrapositively predicts for a [non-integrable](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-l1) variable.

**Exercise 23.11 ★★.**

(Stable laws in embryo) Let $(X_n)$ be i.i.d. standard Cauchy ([Exercise 23.10](#exo-b3-clt-10)). (a) Show that for any $a, b > 0$, $aX_1 + bX_2$ has the law of $(a + b)X_1$: the Cauchy family is *strictly stable* of index $1$. (b) Show that the [Gaussian](#def-b3-clt-gaussianvector) family is strictly stable of index $2$: $aX_1 + bX_2 \sim \sqrt{a^2 + b^2}\,X_1$ for $X_i$ i.i.d. $\mathcal N(0,1)$. (c) Explain, via [characteristic functions](#def-b3-clt-cf) of the form $\eu^{-c\abs\xi^\alpha}$, why index-$\alpha$ stability forces the normalization $n^{1/\alpha}$ for sums, and what this says about the basins of attraction of the CLT: which i.i.d. sums can converge, after affine normalization, to a Cauchy law rather than a [Gaussian](#def-b3-clt-gaussianvector)?

**Solution of Exercise 23.11.**

(a) $\varphi_{aX_1 + bX_2}(\xi) =
\eu^{-a\abs\xi}\eu^{-b\abs\xi} = \eu^{-(a+b)\abs\xi} =
\varphi_{(a+b)X_1}(\xi)$ ([independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) and [Exercise 23.10](#exo-b3-clt-10)); injectivity identifies the laws.

(b) $\varphi_{aX_1+bX_2}(\xi) = \eu^{-a^2\xi^2/2}
\eu^{-b^2\xi^2/2} = \eu^{-(a^2+b^2)\xi^2/2}$: the law of $\sqrt{a^2+b^2}\,X_1$.

(c) If $\varphi_X(\xi) = \eu^{-c\abs\xi^\alpha}$, then $S_n = X_1 + \dots + X_n$ has $\varphi_{S_n} =
\eu^{-cn\abs\xi^\alpha}$, and $S_n/n^{1/\alpha}$ has $\varphi(\xi) = \eu^{-c\abs\xi^\alpha}$ again: exact self-reproduction under the $n^{1/\alpha}$ scaling — $\sqrt n$ for the [Gaussian](#def-b3-clt-gaussianvector) ($\alpha = 2$), $n$ itself for Cauchy ($\alpha = 1$, [Exercise 23.10](#exo-b3-clt-10)(b)). A sum of i.i.d. variables can only converge (after affine normalization) to a law that is stable under such convolutions; the CLT says finite variance forces the [Gaussian](#def-b3-clt-gaussianvector) basin, and the Cauchy basin is reserved for laws with tails so heavy that $\E X^2 = \infty$ and even $\E\abs
X = \infty$ — e.g. sums of Cauchy variables themselves. Universality has several islands, indexed by the tail exponent $\alpha \in \intoc02$.

**Exercise 23.12 ★★.**

(The empirical distribution function) Let $(X_n)$ be i.i.d. with distribution function $F$, and $F_n(t) =
\frac1n\#\{k \leq n : X_k \leq t\}$. (a) Fix $t$. Show that $n F_n(t) \sim \mathcal B(n, F(t))$, that $F_n(t) \to F(t)$ a.s. ([Theorem 22.13](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-slln)), and that

$$
\sqrt n\,\bigl(F_n(t) - F(t)\bigr) \Longrightarrow
\mathcal N\bigl(0,\ F(t)(1 - F(t))\bigr) .
$$

(b) At which $t$ is the asymptotic variance maximal? Interpret: the median is where an empirical distribution is hardest to pin down. (c) For $F$ [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity), show that the law of $\sup_t\abs{F_n(t) - F(t)}$ does not depend on $F$ *(reduce to uniform variables via [Exercise 22.1](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#exo-b3-probability-1))* — the distribution-free miracle behind the Kolmogorov–Smirnov test; no computation of that law is asked.

**Solution of Exercise 23.12.**

(a) The indicators $\mathbf 1_{X_k \leq t}$ are i.i.d. Bernoulli of parameter $p = F(t)$: their sum $nF_n(t)$ is binomial $\mathcal B(n, p)$; the strong law gives $F_n(t)
\to p$ a.s., and the CLT ([Theorem 23.8](#thm-b3-clt-clt)) applied to the same indicators (variance $p(1-p)$) gives the stated [Gaussian](#def-b3-clt-gaussianvector) limit.

(b) $p(1 - p)$ is maximal at $p = \frac12$, i.e. where $F(t) = \frac12$: at the *median*. Estimating tail probabilities is asymptotically easy (variance $\to 0$ as $p \to 0, 1$); the median region carries the largest statistical noise — the empirical curve wobbles most in its middle.

(c) For [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $F$, the variables $U_k = F(X_k)$ are i.i.d. uniform on $\intoo01$ ([Exercise 22.1](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#exo-b3-probability-1)), and monotonicity of $F$ gives, writing $G_n$ for the empirical distribution function of the $U_k$:

$$
\sup_{t\in\R}\,\abs{F_n(t) - F(t)}
= \sup_{u \in \operatorname{im}F}\,\abs{G_n(u) - u}
= \sup_{u\in\intcc01}\abs{G_n(u) - u} :
$$

the first equality because $\{X_k \leq t\} = \{U_k \leq
F(t)\}$ up to null events (monotonicity; strict inequality can fail only on the flat parts of $F$, where both sides are unchanged), and the second because a [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $F$, running from $0$ to $1$, attains every value of $\intoo01$ (intermediate value theorem), and the endpoints add nothing ($G_n(0) - 0 = 0$ and $G_n(1) - 1 = 0$). The right side involves only uniforms: one law for all $F$ — so a single table of critical values (that of Kolmogorov’s distribution) tests *any* [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) model against data.

## 23.6 Problem: Lindeberg’s proof of the CLT, with a rate

**Problem 23.1.**

Weekend problem — the replacement method

Lindeberg (1922) proved the central limit theorem by an idea of disarming simplicity: *swap the summands one at a time for [Gaussians](#def-b3-clt-gaussianvector)* and control each swap by a Taylor expansion. The method needs no Fourier analysis, produces an explicit error rate, and today powers universality proofs across probability theory. Let $(X_i)$ be i.i.d., centered, $\V(X_1) = 1$, with $\beta = \E\abs{X_1}^3 < \infty$; let $(N_i)$ be i.i.d. $\mathcal N(0,1)$, [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of the $X_i$ (existence: [Theorem 22.6](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-existence)). Set

$$
T_n = \frac{X_1 + \dots + X_n}{\sqrt n},
\qquad
G_n = \frac{N_1 + \dots + N_n}{\sqrt n} \sim \mathcal N(0,1).
$$

**Part I — The swapping identity.** Fix $f
\in \mathcal C^3_b(\R)$ (three bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) derivatives; $M_3 = \sup\abs{f'''}$). For $0 \leq i \leq n$ define the hybrid sums

$$
H_i = \frac{X_1 + \dots + X_i + N_{i+1} + \dots +
N_n}{\sqrt n},
$$

so $H_n = T_n$ and $H_0 = G_n$.

1. Write $H_i = W_i + \frac{X_i}{\sqrt n}$ and $H_{i-1}  = W_i + \frac{N_i}{\sqrt n}$ with $W_i =  \frac{1}{\sqrt n}\bigl(\sum_{j<i}X_j +  \sum_{j>i}N_j\bigr)$ , and note that $W_i$ is [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of the pair $(X_i, N_i)$ . Justify.
2. Taylor with integral or Lagrange remainder: for any real $w, h$: $$\Bigl|f(w + h) - f(w) - f'(w)h -  \tfrac12f''(w)h^2\Bigr| \leq  \frac{M_3\,\abs h^3}{6} .$$
3. Apply question 2 twice ($h = \frac{X_i}{\sqrt n}$ and $h = \frac{N_i}{\sqrt n}$ at $w = W_i$), take [expectations](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space), and use [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) plus the matching of the first two moments of $X_i$ and $N_i$ to show $$\bigl|\E f(H_i) - \E f(H_{i-1})\bigr|  \leq \frac{M_3}{6}\cdot  \frac{\beta + \gamma}{n^{3/2}},  \qquad \gamma = \E\abs{N_1}^3 =  \frac{2\sqrt2}{\sqrt\pi} .$$
4. Telescope over $i$ and conclude the *Lindeberg bound*: $$\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq  \frac{M_3\,(\beta + \gamma)}{6\,\sqrt n} .$$

**Part II — From smooth $f$ to the CLT.**

5. Show that $\E f(T_n) \to \E f(N)$ for every $f \in  \mathcal C_b^3$ , and upgrade to all bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f$ : given such $f$ and $\varepsilon$ , construct $f_\varepsilon \in \mathcal C^3_b$ with $\norm{f - f_\varepsilon}_\infty \leq \varepsilon$ on a large interval — e.g. convolve $f$ with a $\mathcal C^\infty$ bump ( [Theorem 12.9](https://one-course.com/books/math/5/en/chapter/12-the-lp-spaces#thm-b3-lp-regularization) ) — and handle the tails by tightness ( $\V(T_n) = 1$ and Chebyshev). Conclude $T_n \Rightarrow \mathcal N(0, 1)$ : the central limit theorem, re-proved.
6. Where did the proof use that the $X_i$ are *identically* distributed? Show that it barely did: state and prove the version for [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) , centered, non-identical $X_i$ with $\sum_i\V(X_i) = s_n^2$ and third moments, obtaining the error $\frac{M_3}{6s_n^3}\sum_i\bigl(\E\abs{X_i}^3  + \V(X_i)^{3/2}\gamma\bigr)$ — Lindeberg’s true theorem in its Lyapunov form.

**Part III — Quantitative dividends.**

7. (Distribution functions) Let $t \in \R$ and approximate $\mathbf 1_{\intoc{-\infty}t}$ above and below by $\mathcal C^3_b$ ramps of width $\delta$ (construct them, with $M_3 = O(\delta^{-3})$). Combining with Part I, derive the two-term bound $$\sup_{t\in\R}\,\bigl|\P(T_n \leq t) -  \Phi(t)\bigr| \;\leq\;  \frac{C_1(\beta + \gamma)}{\delta^3\sqrt n} +  C_2\,\delta  \qquad (\text{every } \delta > 0),$$ with explicit constants (the $C_2\delta$ term uses that $\Phi$ has [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) bounded by $\frac1{\sqrt{2\pi}}$), and optimize $\delta \sim  n^{-1/8}$ to obtain a uniform rate of order $n^{-1/8}$. (The optimal $n^{-1/2}$ — Berry–Esseen — needs finer tools; the point is an *explicit* rate from elementary swapping.)
8. (De Moivre–Laplace, quantified) Specialize to $X_i = 2B_i - 1$ (signs of fair coins): compare the conclusion with the local estimate of [Problem 11.1](https://one-course.com/books/math/5/en/chapter/11-product-measures-fubini-change-of-variables#pb-b3-product-1) , question 7 — what does each method give that the other does not?
9. (Universality) Explain in a paragraph why the replacement method shows more than the CLT: any statistic of the form $\E f(\text{sum})$ with $f$ smooth is insensitive, at order $n^{-1/2}$ , to the *entire law* of the summands beyond its first two moments — the “invariance principle” that underlies modern universality results (random matrices, random polynomials), of which the CLT is the first instance.

**Part IV — Smoothing, pushed: better rates.** The loss from $n^{-1/2}$ (smooth $f$) to $n^{-1/8}$ (distribution functions) came from charging $f'''$ in sup norm. The hybrids can repair part of it: they contain [Gaussian](#def-b3-clt-gaussianvector) summands, and [Gaussians](#def-b3-clt-gaussianvector) *smooth*.

10. (A hidden [Gaussian](#def-b3-clt-gaussianvector)) For $1 \leq i \leq n - 1$, $h =  \frac{X_i}{\sqrt n}$ or $\frac{N_i}{\sqrt n}$, and $\theta \in \intcc01$, write $W_i + \theta h = A +  Z$ with $Z = \frac{N_{i+1} + \dots + N_n}{\sqrt  n}$. Show that $Z \sim \mathcal N\bigl(0,  \frac{n-i}n\bigr)$ is [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of the pair $(A,  h)$, and deduce, for every [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $g \in  L^1(\R)$, $$\E\bigl[\abs h^3\,\abs{g(W_i + \theta h)}\bigr]  \;\leq\; \sqrt{\frac{n}{2\pi(n - i)}}\;  \norm{g}_{L^1}\;\E\abs h^3 .$$
11. Combine question 10 with the integral form of the Taylor remainder, $$f(w + h) = f(w) + f'(w)h + \tfrac12f''(w)h^2 +  \int_0^1\frac{(1 - \theta)^2}2\,f'''(w + \theta  h)\,h^3\,\dd\theta,$$ to redo questions 3–4: for $f \in \mathcal C^3_b$ with moreover $f''' \in L^1(\R)$, $$\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq  \frac{\beta + \gamma}{3\sqrt{2\pi}}\cdot  \frac{\norm{f'''}_{L^1}}{\sqrt n} +  \frac{M_3(\beta + \gamma)}{6\,n^{3/2}}$$ *(question 10 handles the swaps $i \leq n - 1$ — use $\sum_{m=1}^{n-1}m^{-1/2} \leq 2\sqrt n$ — and question 3’s crude bound handles the last one)*. Check that question 7’s ramps satisfy $\norm{\psi_\delta'''}_{L^1} = K_1\delta^{-2}$ while $M_3 = K\delta^{-3}$, feed them in, and optimize $\delta$: the uniform distribution-function rate improves to $O(n^{-1/6})$.
12. (Matching one more moment) Assume in addition $\E  X_1^3 = 0$ and $\beta_4 = \E X_1^4 < \infty$ . Compute $\E N_1^3$ and $\E N_1^4$ , expand to fourth order, and prove along the same lines that the distribution-function rate becomes $O(n^{-1/4})$ *(now $\norm{\psi_\delta^{(4)}}_{L^1} =  K_2\delta^{-3}$ and $M_4 = K'\delta^{-4}$; choose $\delta = n^{-1/4}$)* .
13. (The obstruction) Suppose the first $k$ moments of $X_1$ agree with the [Gaussian](#def-b3-clt-gaussianvector) ones ( $k = 2$ always; $k = 3$ exactly when $\E X_1^3 = 0$ ; $k \geq 4$ essentially never, as $\E N_1^4 = 3$ ). Verify that the scheme of questions 10–12 delivers the distribution-function rate $n^{-(k-1)/(2k+2)}$ , by balancing $\delta^{-k}n^{-(k-1)/2}$ against $\delta$ , and observe that the exponent approaches the Berry–Esseen value $\frac12$ only as $k \to  \infty$ . Explain in a few sentences why the swapping method saturates: each swap is charged in absolute value, whereas the Fourier route (Esseen’s smoothing inequality) exploits the oscillation of the characteristic-function difference and reaches $C\beta n^{-1/2}$ with three moments only.

**Part V — Two dimensions: the multidimensional CLT, by swapping.** Now let the $X_i$ be i.i.d. centered random *vectors* of $\R^2$ with covariance matrix $\Sigma$ and $\beta' = \E\norm{X_1}^3 <
\infty$ (Euclidean norm).

14. ( [Gaussian vectors](#def-b3-clt-gaussianvector) , to order) Diagonalize $\Sigma =  PDP^{\mathsf T}$ ( [Exercise 20.8](https://one-course.com/books/math/5/en/chapter/20-submanifolds-of-rn#exo-b3-submanifolds-8) ) and set $C = P\sqrt DP^{\mathsf T}$ . For $Z = (Z^1,  Z^2)$ a pair of [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) standard [Gaussians](#def-b3-clt-gaussianvector) ( [Theorem 22.6](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-existence) ), show that $N  = CZ$ is a [Gaussian vector](#def-b3-clt-gaussianvector) ( [Definition 23.10](#def-b3-clt-gaussianvector) ) of mean $0$ , covariance $\Sigma$ , with $\gamma' = \E\norm N^3 <  \infty$ ; and that $G_n = \frac{N_1 + \dots +  N_n}{\sqrt n}$ has law $\mathcal N(0, \Sigma)$ *exactly* for i.i.d. copies $N_i$ .
15. (Taylor in two variables) For $f \colon \R^2 \to  \R$ of class $\mathcal C^3$ with $M_3 =  \max_{\abs\alpha = 3}\sup\abs{\partial^\alpha f} <  \infty$, prove $$\Bigl|f(w + h) - f(w) - \langle\nabla f(w),  h\rangle - \tfrac12\langle h, D^2f(w)\,h\rangle  \Bigr| \leq \frac{M_3}6\,\bigl(\abs{h_1} +  \abs{h_2}\bigr)^3 \leq \frac{\sqrt2\,M_3}3\,  \norm h^3$$ *(study $t \mapsto f(w + th)$ on $\intcc01$)*.
16. (The CLT in $\R^2$) Run the replacement scheme on the vector hybrids $H_i$: show that the first- and second-order terms cancel (means and covariances match), telescope, and upgrade as in question 5 (tightness from $\E\norm{T_n}^2 =  \operatorname{tr}\Sigma$; mollification now in $\R^2$, [Theorem 12.9](https://one-course.com/books/math/5/en/chapter/12-the-lp-spaces#thm-b3-lp-regularization)) to conclude: for every bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f \colon  \R^2 \to \R$, $$\E\,f\Bigl(\frac{X_1 + \dots + X_n}{\sqrt n}\Bigr)  \longrightarrow \E\,f(N), \qquad N \sim \mathcal  N(0, \Sigma) :$$ [Theorem 23.12](#thm-b3-clt-multiclt) in dimension $2$, with a rate for smooth $f$ and no Fourier analysis.
17. (Cramér–Wold, and a joint fluctuation) Deduce that $\langle t, \frac{S_n}{\sqrt n}\rangle  \Rightarrow \mathcal N(0, t^{\mathsf T}\Sigma t)$ for every fixed $t \in \R^2$. Application: for i.i.d. real $(\xi_i)$, centered, $\E\xi_1^2 = 1$, $\E\xi_1^6 < \infty$ (so that Part V applies to $V_i = (\xi_i, \xi_i^2 - 1)$), show $$\frac1{\sqrt n}\Bigl(\sum_{i\leq n}\xi_i,\  \sum_{i\leq n}(\xi_i^2 - 1)\Bigr) \Longrightarrow  \mathcal N\Bigl(0, \begin{pmatrix} 1 & \E\xi_1^3\\  \E\xi_1^3 & \E\xi_1^4 - 1\end{pmatrix}\Bigr) :$$ empirical mean and empirical second moment fluctuate jointly Gaussianly — independently in the limit if and only if $\E\xi_1^3 = 0$ ([Theorem 23.11](#thm-b3-clt-gaussianvector)).

**Part VI — The delta method.**

18. Let $(\hat\theta_n)$ be [random variables](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) with $\sqrt n(\hat\theta_n - \theta) \Rightarrow  \mathcal N(0, \sigma^2)$ for a real parameter $\theta$, and let $g$ be differentiable at $\theta$. Prove the *delta method*: $$\sqrt n\bigl(g(\hat\theta_n) - g(\theta)\bigr)  \Longrightarrow \mathcal N\bigl(0,  g'(\theta)^2\sigma^2\bigr)$$ *(write $g(x) - g(\theta) = (g'(\theta) +  \eta(x))(x - \theta)$ with $\eta \to 0$ at $\theta$; show $\hat\theta_n \to \theta$, then $\eta(\hat\theta_n) \to 0$, in probability; finish with Slutsky, [Exercise 23.8](#exo-b3-clt-8), and [Exercise 23.4](#exo-b3-clt-4)(b))*.
19. Applications. (a) For i.i.d. real $(\xi_i)$ with mean $\mu$ and variance $\sigma^2$, and $\bar X_n =  \frac1n\sum_{i\leq n}\xi_i$: show $\sqrt n(\bar  X_n^2 - \mu^2) \Rightarrow \mathcal N(0,  4\mu^2\sigma^2)$ when $\mu \neq 0$, and that for $\mu = 0$ the correct statement lives at another scale: $n\bar X_n^2 \Rightarrow \sigma^2N^2$ with $N \sim \mathcal N(0,1)$ (identify the limit’s distribution function). (b) (Variance stabilization) For $\hat p_n$ the success frequency of a $\mathcal B(1, p)$ sample, $p \in \intoo01$: show that $g(p) = \arcsin\sqrt p$ satisfies $$\sqrt n\,\bigl(g(\hat p_n) - g(p)\bigr)  \Longrightarrow \mathcal N\Bigl(0, \frac14\Bigr)$$ *whatever* $p$ — an asymptotic error bar free of the unknown parameter; compare with [Example 23.9](#ex-b3-clt-confidence).

**Part VII — Poisson, by the same method: Le Cam’s theorem.** Replacement knows a second universality class: sums of many [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) *rare* events. For laws on $\N$ the right distance is *total variation*,

$$
d_{\mathrm{TV}}(\mu, \nu) = \sup_{A\subseteq\N}\,
\abs{\mu(A) - \nu(A)} .
$$

20. Show that $d_{\mathrm{TV}}(\mu, \nu) =  \frac12\sum_{k\geq0}\abs{\mu(\{k\}) -  \nu(\{k\})}$ , and prove the coupling bound: for *any* pair $(X, Y)$ of [random variables](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) with laws $\mu$ and $\nu$ on the same space, $d_{\mathrm{TV}}(\mu, \nu) \leq \P(X \neq Y)$ .
21. Compute exactly, for $p \in \intoo01$: $$d_{\mathrm{TV}}\bigl(\mathcal B(1, p), \mathcal  P(p)\bigr) = p\bigl(1 - \eu^{-p}\bigr) \leq p^2 .$$
22. (Le Cam, by swapping) Let $X_i \sim \mathcal B(1,  p_i)$ and $Y_i \sim \mathcal P(p_i)$, the $2n$ variables [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence); $S = X_1 + \dots + X_n$, and recall $Y_1 + \dots + Y_n \sim \mathcal P(\lambda)$ with $\lambda = \sum_ip_i$ ([Exercise 23.1](#exo-b3-clt-1)). Swap one coordinate at a time in the integer hybrids $H_i = Y_1 + \dots + Y_i + X_{i+1} + \dots  + X_n$: show, for every $A \subseteq \N$, $$\abs{\P(H_{i-1} \in A) - \P(H_i \in A)} \leq  d_{\mathrm{TV}}\bigl(\mathcal B(1, p_i), \mathcal  P(p_i)\bigr),$$ and conclude *Le Cam’s inequality*: $$d_{\mathrm{TV}}\bigl(\text{law of } S,\ \mathcal  P(\lambda)\bigr) \leq \sum_{i=1}^np_i^2 .$$
23. Dividends. (a) For $p_i = \frac\lambda n$ : the bound is $\frac{\lambda^2}n$ — the law of rare events ( [Exercise 23.5](#exo-b3-clt-5) ) upgraded to an explicit rate, uniform over all events, and valid for unequal $p_i$ as well. (b) $500$ letters are delivered, each going astray independently with probability $\frac1{500}$ : bound the error of the Poisson model of parameter $1$ , and estimate the probability that no letter goes astray. (c) Close the problem: compare the two universality classes met here — [Gaussian](#def-b3-clt-gaussianvector) (many small spread-out contributions; two moments matched; Taylor) and Poisson (many rare contributions; one mean matched; an exact total-variation coupling) — and the single replacement method behind both.
24. (Relative error and the log transform) Let $(X_n)$ be i.i.d., positive, mean $\mu > 0$, variance $\sigma^2$, and $\bar X_n$ the empirical mean. Show by the delta method that $$\sqrt n\,\bigl(\ln\bar X_n - \ln\mu\bigr)  \Longrightarrow  \mathcal N\Bigl(0,\ \frac{\sigma^2}{\mu^2}\Bigr) :$$ the asymptotic parameter of $\ln\bar X_n$ is the *coefficient of variation* $\sigma/\mu$ — relative, scale-free error. Deduce a $95\%$ confidence interval for $\mu$ of the multiplicative form $\bar X_n\cdot\eu^{\pm1.96\,\sigma/(\mu\sqrt  n)}$, and explain when it is preferable to the additive one.
25. (The third moment steers the error) For $X \sim$ Bernoulli( $p$ ) centered, compute $\E\bigl[(X -  p)^3\bigr] = p(1-p)(1-2p)$ . Using Part IV’s analysis (the swap error is driven by third moments), explain why the normal approximation of $\mathcal B(n, p)$ is asymmetric for $p \neq \frac12$ — overshooting on one side, undershooting on the other — and why $p = \frac12$ enjoys the faster matched-moment rate. Verify the sign of the skew numerically on $\mathcal  B(20, 0.1)$ against $\mathcal N(2, 1.8)$ : compare $\P(S = 0) = 0.9^{20}$ with the [Gaussian](#def-b3-clt-gaussianvector) mass of $\intoo{-\infty}{0.5}$ .

**Solution of Problem 23.1.**

**1.** The family $(X_1, \dots, X_n, N_1, \dots, N_n)$ is [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence): the two blocks are [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of each other by construction and each block is i.i.d. $W_i$ is a [measurable function](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#def-b3-lebesgue-measurable) of the variables $(X_j)_{j<i}$ and $(N_j)_{j>i}$ only, all distinct from $X_i$ and $N_i$: by the coalition principle ([Theorem 22.5](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-independence)), $W_i$ is [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of the pair $(X_i, N_i)$. The decompositions $H_i = W_i + \frac{X_i}{\sqrt n}$ and $H_{i-1} = W_i +
\frac{N_i}{\sqrt n}$ are immediate from the definitions: passing from $H_i$ to $H_{i-1}$ swaps the single summand $X_i$ for $N_i$.

**2.** Taylor–Lagrange at order $3$: there is $c$ between $w$ and $w + h$ with $f(w + h) = f(w) + f'(w)h +
\frac12f''(w)h^2 + \frac16f'''(c)h^3$, and $\abs{f'''(c)}
\leq M_3$ gives the bound.

**3.** Subtracting the two expansions at the common base point $w = W_i$:

$$
f(H_i) - f(H_{i-1}) = f'(W_i)\,\frac{X_i - N_i}{\sqrt n} +
\frac{f''(W_i)}{2}\,\frac{X_i^2 - N_i^2}{n} + R_i,
\qquad
\abs{R_i} \leq \frac{M_3}{6}\cdot
\frac{\abs{X_i}^3 + \abs{N_i}^3}{n^{3/2}} .
$$

Take [expectations](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space). By question 1, $f'(W_i)$ and $f''(W_i)$ are [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of $(X_i, N_i)$, so the mixed [expectations](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) factor:

$$
\begin{align*}
\E\Bigl[f'(W_i)\,\frac{X_i - N_i}{\sqrt n}\Bigr] &=
\E\bigl[f'(W_i)\bigr]\,\frac{\E X_i - \E N_i}{\sqrt n} = 0,
\\
\E\Bigl[f''(W_i)\,\frac{X_i^2 - N_i^2}{n}\Bigr] &=
\E\bigl[f''(W_i)\bigr]\,\frac{1 - 1}{n} = 0 :
\end{align*}
$$

the first two moments of $X_i$ and $N_i$ *match*, and only the remainder survives:

$$
\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq \E\abs{R_i} \leq
\frac{M_3}{6}\cdot\frac{\beta + \gamma}{n^{3/2}} .
$$

The [Gaussian](#def-b3-clt-gaussianvector) third moment: $\gamma = \E\abs{N_1}^3 =
2\int_0^\infty x^3\,\frac{\eu^{-x^2/2}}{\sqrt{2\pi}}\,\dd x
= \frac{2}{\sqrt{2\pi}}\int_0^\infty 2u\,\eu^{-u}\dd u =
\frac{4}{\sqrt{2\pi}} = \frac{2\sqrt2}{\sqrt\pi}$ (substitution $u = x^2/2$, then $\Gamma(2) = 1$).

**4.** Telescoping $\E f(T_n) - \E f(G_n) =
\sum_{i=1}^n\bigl(\E f(H_i) - \E f(H_{i-1})\bigr)$ and applying question 3 to each of the $n$ terms:

$$
\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq
n \cdot \frac{M_3(\beta + \gamma)}{6\,n^{3/2}} =
\frac{M_3\,(\beta + \gamma)}{6\,\sqrt n} .
$$

**5.** $G_n$ is $\mathcal N(0,1)$ *exactly* for every $n$ (a normalized sum of [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) standard [Gaussians](#def-b3-clt-gaussianvector), [Exercise 23.3](#exo-b3-clt-3)), so $\E f(G_n) = \E f(N)$ and question 4 reads $\abs{\E f(T_n) - \E f(N)} \leq
\frac{M_3(\beta+\gamma)}{6\sqrt n} \to 0$ for $f \in
\mathcal C^3_b$. *Upgrade.* Let $f$ be bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity), $M = \sup\abs f$, $\varepsilon > 0$. Choose $A
\geq 1$ with $\frac1{A^2} \leq \varepsilon$: Chebyshev with $\V(T_n) = 1$ gives $\P(\abs{T_n} > A) \leq \varepsilon$ for all $n$, and likewise $\P(\abs N > A) \leq \varepsilon$. Let $\chi$ be $\mathcal C^\infty$ with $\mathbf 1_{\intcc{-A}A} \leq \chi \leq
\mathbf 1_{\intcc{-A-1}{A+1}}$ (a smooth plateau, built by mollifying $\mathbf 1_{\intcc{-A-\frac12}{A+\frac12}}$, [Theorem 12.9](https://one-course.com/books/math/5/en/chapter/12-the-lp-spaces#thm-b3-lp-regularization)); $g = f\chi$ is [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) with [compact](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-compact) support, hence uniformly [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity), so its mollification $g_\eta = g * \rho_\eta$ is $\mathcal
C^\infty$ with bounded derivatives of all orders and $\norm{g - g_\eta}_\infty \leq \varepsilon$ for $\eta$ small enough. For $T = T_n$ or $N$, since $f = g$ on $\intcc{-A}A$ and $\abs{f - g} \leq 2M$ everywhere:

$$
\bigl|\E f(T) - \E g_\eta(T)\bigr| \leq
\E\abs{(f - g)(T)} + \norm{g - g_\eta}_\infty
\leq 2M\,\P(\abs T > A) + \varepsilon \leq
(2M + 1)\,\varepsilon .
$$

Combining with $\E g_\eta(T_n) \to \E g_\eta(N)$ (question 4 applies: $g_\eta \in \mathcal C^3_b$):

$$
\limsup_n\;\bigl|\E f(T_n) - \E f(N)\bigr| \leq
2(2M + 1)\,\varepsilon ,
$$

and $\varepsilon$ was arbitrary: $\E f(T_n) \to \E f(N)$ for every bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f$, i.e. $T_n \Rightarrow
\mathcal N(0,1)$.

**6.** Identical distribution entered only through one sentence: “$X_i$ and $N_i$ have the same first two moments”. So let $X_1, \dots, X_n$ be [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence), centered, with variances $\sigma_i^2$ and finite third moments, $s_n^2 = \sum_i\sigma_i^2 > 0$, and take $N_i \sim
\mathcal N(0, \sigma_i^2)$ [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of everything. Define the hybrids with normalization $s_n$: $H_i =
\frac1{s_n}(\sum_{j\leq i}X_j + \sum_{j>i}N_j)$. In the $i$-th swap, $\E X_i = \E N_i = 0$ and $\E X_i^2 = \E N_i^2
= \sigma_i^2$ again kill the $f'$ and $f''$ terms, and the remainder gives (using $\E\abs{N_i}^3 = \sigma_i^3\gamma$ by scaling):

$$
\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq
\frac{M_3}{6\,s_n^3}\bigl(\E\abs{X_i}^3 +
\sigma_i^3\gamma\bigr) .
$$

Telescoping:

$$
\Bigl|\E f\Bigl(\frac{X_1 + \dots + X_n}{s_n}\Bigr) -
\E f(N)\Bigr| \leq \frac{M_3}{6\,s_n^3}\sum_{i=1}^n
\Bigl(\E\abs{X_i}^3 + \V(X_i)^{3/2}\,\gamma\Bigr) .
$$

Since $\sigma_i^3 = (\E X_i^2)^{3/2} \leq \E\abs{X_i}^3$ (the power–mean inequality, i.e. Jensen for $t \mapsto
t^{3/2}$ applied to $X_i^2$), the right side is at most $\frac{M_3(1 + \gamma)}{6}\cdot
\frac{\sum_i\E\abs{X_i}^3}{s_n^3}$: under *Lyapunov’s condition* $\frac1{s_n^3}\sum_i\E\abs{X_i}^3 \to 0$, the normalized sums converge in law to $\mathcal N(0,1)$ — the CLT without identical distribution.

**7.** Let $\rho \in \mathcal C^\infty_c(\intoo01)$ with $\int\rho = 1$ and set $\psi(x) = \int_x^1\rho(s)\dd
s$: $\psi$ is $\mathcal C^\infty$, nonincreasing, $\psi = 1$ on $\R_-$, $\psi = 0$ on $\intco1\infty$; let $K =
\norm{\psi'''}_\infty$. For $t \in \R$ and $\delta > 0$ define $\psi_\delta(x) = \psi\bigl(\frac{x - t}\delta\bigr)$ and $\tilde\psi_\delta(x) = \psi\bigl(\frac{x - t}\delta +
1\bigr)$: these are $\mathcal C^3_b$ with third derivative bounded by $K/\delta^3$, and

$$
\mathbf 1_{\intoc{-\infty}{t-\delta}} \leq
\tilde\psi_\delta \leq \mathbf 1_{\intoc{-\infty}t} \leq
\psi_\delta \leq \mathbf 1_{\intoc{-\infty}{t+\delta}} .
$$

Upper bound: by question 4 applied to $\psi_\delta$ (with $M_3 = K/\delta^3$),

$$
\P(T_n \leq t) \leq \E\psi_\delta(T_n) \leq
\E\psi_\delta(N) + \frac{K(\beta +
\gamma)}{6\,\delta^3\sqrt n}
\leq \Phi(t + \delta) + \frac{K(\beta +
\gamma)}{6\,\delta^3\sqrt n}
\leq \Phi(t) + \frac{\delta}{\sqrt{2\pi}} +
\frac{K(\beta + \gamma)}{6\,\delta^3\sqrt n},
$$

because $\Phi$ is Lipschitz with constant $\frac1{\sqrt{2\pi}}$ (its [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) is bounded by $\frac1{\sqrt{2\pi}}$). The symmetric lower bound via $\tilde\psi_\delta$ gives the two-term estimate

$$
\sup_{t\in\R}\,\bigl|\P(T_n \leq t) - \Phi(t)\bigr| \leq
\frac{K(\beta + \gamma)}{6}\cdot\frac{1}{\delta^3\sqrt n} +
\frac{\delta}{\sqrt{2\pi}}
\qquad(\delta > 0\ \text{arbitrary}).
$$

The two terms balance when $\delta^{-3}n^{-1/2} \asymp
\delta$, i.e. $\delta = n^{-1/8}$: both are then $O(n^{-1/8})$, an explicit uniform rate valid for every $n$. (The optimal Berry–Esseen rate $C\beta/\sqrt n$ requires the Fourier smoothing inequality; swapping trades sharpness for [complete](https://one-course.com/books/math/5/en/chapter/7-complete-spaces-baire-ascoli-stoneweierstrass#def-b3-complete-complete) elementarity.)

**8.** For $X_i = 2B_i - 1$ (fair signs): centered, variance $1$, and $\abs{X_i} = 1$ so $\beta = 1$. Question 7 then bounds $\sup_t\abs{\P(\frac{S_n}{\sqrt n} \leq t) -
\Phi(t)}$ explicitly and uniformly for *every* finite $n$ — a global, non-asymptotic statement about the distribution function. The local estimate of [Problem 11.1](https://one-course.com/books/math/5/en/chapter/11-product-measures-fubini-change-of-variables#pb-b3-product-1), question 7, gives instead the exact asymptotics of an individual atom, $\P(S_{2n} = 2k) \sim
\frac{\eu^{-k^2/n}}{\sqrt{\pi n}}$: it resolves probabilities of size $n^{-1/2}$, far below question 7’s $n^{-1/8}$ resolution, but it is pointwise, asymptotic (no explicit error at fixed $n$) and tied to this particular lattice law. Local precision versus global uniformity: the two methods are complementary, and summing the local estimate over $k \in \intint{a\sqrt n}{b\sqrt n}$ recovers de Moivre–Laplace on intervals — with a sharper rate, but only for this law.

**9.** The swapping argument used nothing about the law of the $X_i$ beyond $\E X_i = 0$, $\E X_i^2 = 1$ and the finiteness of $\E\abs{X_i}^3$: had we replaced the [Gaussians](#def-b3-clt-gaussianvector) $N_i$ by any other i.i.d. family with the same first two moments and finite third moment, the same telescoping would bound $\abs{\E f(\text{sum}_X) - \E f(\text{sum}_Y)}$ by $O(n^{-1/2})$ for every smooth $f$. Smooth statistics of large [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) sums are therefore *universal*: up to a quantified error, they depend on the summands’ law only through two numbers. This is the invariance principle: prove a limit theorem for the most computable law (the [Gaussian](#def-b3-clt-gaussianvector), where everything is exact), then transfer it to all laws by swapping. The same scheme — with sums replaced by more elaborate functionals — drives Wigner’s semicircle law for random matrices, the universality of roots of random polynomials, and much of modern probability; the central limit theorem is its first and simplest instance.

**10.** $Z$ is a Borel function of $(N_{i+1}, \dots,
N_n)$ only, while $A = W_i + \theta h - Z$ and $h$ are functions of the remaining variables of the [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) family $(X_1, \dots, X_n, N_1, \dots, N_n)$: by the coalition principle ([Theorem 22.5](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#thm-b3-probability-independence)), $Z$ is [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of $(A, h)$. As a sum of the [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) $N_j/\sqrt n \sim \mathcal N(0, \frac1n)$, $Z \sim \mathcal
N(0, s^2)$ with $s^2 = \frac{n-i}n$ ([Exercise 23.3](#exo-b3-clt-3)), with [density](https://one-course.com/books/math/5/en/chapter/10-the-lebesgue-integral#ex-b3-lebesgue-gamma) bounded by $\frac1{s\sqrt{2\pi}}$. The law of $((A, h), Z)$ is the product of the two marginal laws, so Tonelli (transfer) freezes the first block: with $G(a) =
\E\abs{g(a + Z)} = \int\abs{g(a + z)}\,\varphi_s(z)\,\dd z
\leq \frac{\norm g_{L^1}}{s\sqrt{2\pi}}$ for every $a$,

$$
\E\bigl[\abs h^3\abs{g(A + Z)}\bigr] =
\E\bigl[\abs h^3\,G(A)\bigr] \leq
\frac{\norm g_{L^1}}{s\sqrt{2\pi}}\,\E\abs h^3
= \sqrt{\frac{n}{2\pi(n-i)}}\,\norm g_{L^1}\,\E\abs h^3 .
$$

**11.** The integral form of Taylor’s formula follows by integrating $f(w + h) - f(w) =
h\int_0^1f'(w + \theta h)\,\dd\theta$ by parts twice in $\theta$. Taking [expectations](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-space) in the $i$-th swap, the orders $0, 1, 2$ cancel exactly as in question 3, and the two remainders (for $h = X_i/\sqrt n$ and $N_i/\sqrt n$) are bounded, for $i \leq n - 1$, by question 10 with $g = f'''$:

$$
\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq
\int_0^1\frac{(1-\theta)^2}2\,\dd\theta\;
\sqrt{\frac{n}{2\pi(n-i)}}\,\norm{f'''}_{L^1}
\frac{\beta + \gamma}{n^{3/2}}
= \frac{\beta + \gamma}{6\,n^{3/2}}
\sqrt{\frac{n}{2\pi(n-i)}}\,\norm{f'''}_{L^1} .
$$

Summing, with $\sum_{i=1}^{n-1}\sqrt{\frac n{n-i}} =
\sqrt n\sum_{m=1}^{n-1}m^{-1/2} \leq 2n$, and adding question 3’s bound for the last swap ($i = n$, no [Gaussian](#def-b3-clt-gaussianvector) left):

$$
\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq
\frac{\beta + \gamma}{3\sqrt{2\pi}}\cdot
\frac{\norm{f'''}_{L^1}}{\sqrt n} +
\frac{M_3(\beta + \gamma)}{6\,n^{3/2}} .
$$

Ramps: $\psi_\delta'''(x) =
\delta^{-3}\psi'''\bigl(\frac{x - t}\delta\bigr)$, so $M_3 = K\delta^{-3}$ with $K = \norm{\psi'''}_\infty$ and $\norm{\psi_\delta'''}_{L^1} = \delta^{-2}
\norm{\psi'''}_{L^1} = K_1\delta^{-2}$ (substitution). The sandwich of question 7 then gives

$$
\sup_t\,\bigl|\P(T_n \leq t) - \Phi(t)\bigr| \leq
\frac{K_1(\beta + \gamma)}{3\sqrt{2\pi}}\cdot
\frac1{\delta^2\sqrt n} +
\frac{K(\beta + \gamma)}{6}\cdot
\frac1{\delta^3n^{3/2}} + \frac\delta{\sqrt{2\pi}} .
$$

At $\delta = n^{-1/6}$ the first and third terms are $O(n^{-1/6})$ and the middle one $O(n^{-1})$: a uniform rate $O(n^{-1/6})$, strictly better than question 7’s $n^{-1/8}$ — the [Gaussian](#def-b3-clt-gaussianvector) half of the hybrid did the extra smoothing.

**12.** $\E N_1^3 = 0$ (odd integrand), and integration by parts gives $\E N_1^4 = 3\,\E N_1^2 = 3$ ($\int x^3\cdot x\varphi(x)\dd x =
3\int x^2\varphi$). For $f$ of class $\mathcal C^4$ with bounded derivatives, expand each swap to fourth order: the third-order terms carry the factor $\E X_i^3 - \E N_i^3 =
0$ ([independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) factors them as in question 3), so only the fourth-order remainder $\int_0^1\frac{(1-\theta)^3}6f^{(4)}(w + \theta
h)h^4\dd\theta$ survives, with $\int_0^1
\frac{(1-\theta)^3}6\dd\theta = \frac1{24}$ and $\E h^4 =
\beta_4n^{-2}$ or $3n^{-2}$. Question 10 (with $g =
f^{(4)}$) bounds the swaps $i \leq n - 1$, and summing as in question 11:

$$
\bigl|\E f(T_n) - \E f(G_n)\bigr| \leq
\frac{\beta_4 + 3}{12\sqrt{2\pi}}\cdot
\frac{\norm{f^{(4)}}_{L^1}}{n} +
\frac{M_4(\beta_4 + 3)}{24\,n^2} .
$$

With $\norm{\psi_\delta^{(4)}}_{L^1} = K_2\delta^{-3}$ and $M_4 = K'\delta^{-4}$, the distribution-function bound becomes $C\delta^{-3}n^{-1} + C'\delta^{-4}n^{-2} +
\frac\delta{\sqrt{2\pi}}$; at $\delta = n^{-1/4}$ the outer terms are $O(n^{-1/4})$ and the middle $O(n^{-1})$: rate $O(n^{-1/4})$.

**13.** With $k$ matched moments the surviving remainder per swap is of order $\E\abs h^{k+1} \asymp
n^{-(k+1)/2}$; the [hidden-Gaussian](#def-b3-clt-gaussianvector) bound charges $\norm{f^{(k+1)}}_{L^1}$ and the sum over swaps contributes the factor $2n$, giving $\asymp\norm{f^{(k+1)}}_{L^1}\,
n^{-(k-1)/2}$ for smooth $f$. Ramps cost $\norm{\psi_\delta^{(k+1)}}_{L^1} \asymp \delta^{-k}$, so the distribution-function error is $\asymp
\delta^{-k}n^{-(k-1)/2} + \delta$, balanced at $\delta =
n^{-(k-1)/(2k+2)}$: rate $n^{-(k-1)/(2k+2)}$, which is $n^{-1/6}$ for $k = 2$, $n^{-1/4}$ for $k = 3$, and tends to $n^{-1/2}$ only as $k \to \infty$ — but $k \geq 4$ would force $\E X_1^4 = 3$ and beyond, i.e. a law that already imitates the [Gaussian](#def-b3-clt-gaussianvector). The saturation is structural: swapping adds $n$ swap errors *in absolute value*, renouncing all cancellation between swaps. The Fourier proof compares [characteristic functions](#def-b3-clt-cf), where the errors appear with their oscillating phases; Esseen’s smoothing inequality converts $\abs{\varphi_{T_n}
- \varphi_N}$, integrated against $\frac{\dd\xi}{\abs\xi}$, into a distribution-function bound at only logarithmic cost, and delivers Berry–Esseen’s $C\beta n^{-1/2}$ from three moments. Replacement trades optimality for robustness — and, as Part VII shows, for portability.

**14.** $\Sigma$ is symmetric positive semidefinite; with $\Sigma = PDP^{\mathsf T}$ ($P$ orthogonal, $D \geq 0$ diagonal, [Exercise 20.8](https://one-course.com/books/math/5/en/chapter/20-submanifolds-of-rn#exo-b3-submanifolds-8)), the symmetric $C =
P\sqrt DP^{\mathsf T}$ satisfies $C^2 = \Sigma$. For any $t
\in \R^2$, $\langle t, CZ\rangle = \langle Ct, Z\rangle =
(Ct)_1Z^1 + (Ct)_2Z^2$ is a linear combination of [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) [Gaussians](#def-b3-clt-gaussianvector), hence [Gaussian](#def-b3-clt-gaussianvector) ([Exercise 23.3](#exo-b3-clt-3)): $N = CZ$ is a [Gaussian vector](#def-b3-clt-gaussianvector); its mean is $0$ and its covariance $\E[NN^{\mathsf T}] =
C\,\E[ZZ^{\mathsf T}]\,C^{\mathsf T} = CC^{\mathsf T} =
\Sigma$. Moments: $\norm N^3 \leq (\abs{N_1} +
\abs{N_2})^3 \leq 4(\abs{N_1}^3 + \abs{N_2}^3)$ (convexity of $x^3$ on $\R_+$), and each coordinate is a real [Gaussian](#def-b3-clt-gaussianvector) with moments of all orders ([Exercise 11.10](https://one-course.com/books/math/5/en/chapter/11-product-measures-fubini-change-of-variables#exo-b3-product-10)): $\gamma' < \infty$. Finally each $\langle t, G_n\rangle = \frac1{\sqrt
n}\sum_i\langle t, N_i\rangle$ is a normalized sum of i.i.d. $\mathcal N(0, t^{\mathsf T}\Sigma t)$, hence exactly $\mathcal N(0, t^{\mathsf T}\Sigma t)$: $G_n$ is a [Gaussian vector](#def-b3-clt-gaussianvector) with mean $0$ and covariance $\Sigma$, and its law is $\mathcal N(0, \Sigma)$ ([Definition 23.10](#def-b3-clt-gaussianvector): the law is determined by these data).

**15.** Let $\phi(t) = f(w + th)$, $t \in \intcc01$: $\phi$ is $\mathcal C^3$ with

$$
\phi'''(t) = \sum_{j,k,l\in\{1,2\}}\partial_{jkl}f(w +
th)\,h_jh_kh_l, \qquad \abs{\phi'''(t)} \leq
M_3\Bigl(\sum_j\abs{h_j}\Bigr)^3 = M_3(\abs{h_1} +
\abs{h_2})^3 .
$$

Taylor–Lagrange at order $3$ for $\phi$ between $0$ and $1$ gives the first inequality; Cauchy–Schwarz gives $\abs{h_1} + \abs{h_2} \leq \sqrt2\norm h$, whence the constant $\frac{2\sqrt2M_3}6 = \frac{\sqrt2M_3}3$.

**16.** Define $H_i$ and $W_i$ as in question 1, now in $\R^2$; the coalition argument is unchanged. In the $i$-th swap, the first-order terms give $\sum_j\E[\partial_jf(W_i)]\,(\E X_{i,j} - \E N_{i,j})/
\sqrt n = 0$ and the second-order terms give $\frac1{2n}\sum_{j,k}\E[\partial_{jk}f(W_i)]\,(\Sigma_{jk}
- \Sigma_{jk}) = 0$: means and covariances match. Question 15 bounds the two remainders:

$$
\bigl|\E f(H_i) - \E f(H_{i-1})\bigr| \leq
\frac{\sqrt2M_3}3\cdot\frac{\E\norm{X_i}^3 +
\E\norm{N_i}^3}{n^{3/2}}
= \frac{\sqrt2M_3(\beta' + \gamma')}{3\,n^{3/2}},
$$

and telescoping over the $n$ swaps:

$$
\Bigl|\E f\Bigl(\frac{S_n}{\sqrt n}\Bigr) - \E
f(G_n)\Bigr| \leq \frac{\sqrt2\,M_3(\beta' +
\gamma')}{3\sqrt n},
\qquad G_n \sim \mathcal N(0, \Sigma)\ \text{exactly} .
$$

Upgrade: $\E\norm{T_n}^2 = \E\norm{X_1}^2 =
\operatorname{tr}\Sigma$ (cross terms vanish by [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) and centering), so $\P(\norm{T_n} > A) \leq
\operatorname{tr}\Sigma/A^2$, and likewise for $N$: tightness. Given a bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f$ and $\varepsilon
> 0$, multiply by a smooth plateau $\chi$ equal to $1$ on the ball of radius $A$ and supported in radius $A + 1$ (mollify an indicator in $\R^2$, [Theorem 12.9](https://one-course.com/books/math/5/en/chapter/12-the-lp-spaces#thm-b3-lp-regularization)); $g = f\chi$ is uniformly [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) with [compact](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-compact) support, so its two-dimensional mollification $g_\eta$ is $\mathcal C^\infty$ with bounded derivatives of all orders and $\norm{g - g_\eta}_\infty
\leq \varepsilon$ for small $\eta$. The three-$\varepsilon$ chain of question 5 then transfers verbatim: $\E f(T_n)
\to \E f(N)$ for every bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $f \colon \R^2
\to \R$. This is [Theorem 23.12](#thm-b3-clt-multiclt) for $d = 2$, now proved — swapping sidesteps the two-dimensional Lévy theorem that the chapter had left admitted.

**17.** For bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) $g \colon \R \to \R$, the map $x \mapsto g(\langle t, x\rangle)$ is bounded [continuous](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) on $\R^2$, so question 16 gives $\E g(\langle
t, T_n\rangle) \to \E g(\langle t, N\rangle)$: every projection converges in distribution, and $\langle t,
N\rangle \sim \mathcal N(0, t^{\mathsf T}\Sigma t)$. (This is the easy direction of Cramér–Wold: joint convergence implies convergence of all linear images.) Application: $V_i = (\xi_i, \xi_i^2 - 1)$ are i.i.d. centered vectors ($\E\xi_1^2 = 1$), with covariance entries $\V(\xi_1) =
1$, $\operatorname{Cov}(\xi_1, \xi_1^2 - 1) = \E\xi_1^3$ and $\V(\xi_1^2 - 1) = \E\xi_1^4 - 1$; the third moment $\E\norm{V_1}^3 \leq 4\bigl(\E\abs{\xi_1}^3 +
\E\abs{\xi_1^2 - 1}^3\bigr)$ is finite when $\xi_1 \in
L^6$. Question 16 yields the displayed joint [Gaussian](#def-b3-clt-gaussianvector) limit, and [Theorem 23.11](#thm-b3-clt-gaussianvector)(2): the two limit coordinates are [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) exactly when the covariance $\E\xi_1^3$ vanishes — for symmetric laws, empirical mean and empirical variance decouple asymptotically.

**18.** Write $g(x) - g(\theta) = (g'(\theta) +
\eta(x))(x - \theta)$ where $\eta(x) = \frac{g(x) -
g(\theta)}{x - \theta} - g'(\theta)$ for $x \neq \theta$ and $\eta(\theta) = 0$: differentiability at $\theta$ means precisely $\eta(x) \to 0$ as $x \to \theta$. *Step 1:* $\hat\theta_n \to \theta$ in probability: for $\varepsilon > 0$ and any $A > 0$, eventually $\varepsilon\sqrt n \geq A$, so $\P(\abs{\hat\theta_n -
\theta} > \varepsilon) \leq \P(\abs{\sqrt
n(\hat\theta_n - \theta)} > A) \to \P(\sigma\abs N > A)$ (distribution functions converge at the [continuity](https://one-course.com/books/math/5/en/chapter/6-general-topology#def-b3-topology-continuity) points $\pm A$), and the right side tends to $0$ as $A \to
\infty$. *Step 2:* $\eta(\hat\theta_n) \to 0$ in probability: given $\varepsilon' > 0$, pick $\delta$ with $\abs\eta \leq \varepsilon'$ on $\abs{x - \theta} \leq
\delta$; then $\P(\abs{\eta(\hat\theta_n)} > \varepsilon')
\leq \P(\abs{\hat\theta_n - \theta} > \delta) \to 0$. *Step 3:*

$$
\sqrt n\bigl(g(\hat\theta_n) - g(\theta)\bigr) =
g'(\theta)\,\sqrt n(\hat\theta_n - \theta) +
\eta(\hat\theta_n)\cdot\sqrt n(\hat\theta_n - \theta) .
$$

By Slutsky’s product rule ([Exercise 23.8](#exo-b3-clt-8), with the sequence $\eta(\hat\theta_n) \to 0$ in probability and the convergent-in-law $\sqrt n(\hat\theta_n - \theta)$), the second term converges in law to $0\cdot\mathcal N(0,
\sigma^2) = 0$, hence to $0$ in probability ([Exercise 23.4](#exo-b3-clt-4)(b)); the first converges in law to $g'(\theta)\mathcal N(0, \sigma^2)$ (Slutsky again, or the affine rule for [characteristic functions](#def-b3-clt-cf)); Slutsky’s sum rule assembles them: the limit is $\mathcal N(0,
g'(\theta)^2\sigma^2)$.

**19.** (a) The CLT gives $\sqrt n(\bar X_n - \mu)
\Rightarrow \mathcal N(0, \sigma^2)$; the delta method with $g(x) = x^2$, $g'(\mu) = 2\mu$, gives $\sqrt n(\bar
X_n^2 - \mu^2) \Rightarrow \mathcal N(0, 4\mu^2\sigma^2)$ — degenerate (limit $0$) when $\mu = 0$. In that case the fluctuation lives one scale up: $n\bar X_n^2 =
(\sqrt n\,\bar X_n)^2$, and for $t > 0$

$$
\P\bigl(n\bar X_n^2 \leq t\bigr) = \P\bigl(-\sqrt t \leq
\sqrt n\,\bar X_n \leq \sqrt t\bigr) \longrightarrow
\Phi\Bigl(\frac{\sqrt t}\sigma\Bigr) -
\Phi\Bigl(-\frac{\sqrt t}\sigma\Bigr) = \P(\sigma^2N^2
\leq t) :
$$

$n\bar X_n^2 \Rightarrow \sigma^2N^2$, the square of a [Gaussian](#def-b3-clt-gaussianvector) (a “chi-squared” law) — when the first derivative dies, the second-order term of Taylor dictates a [non-Gaussian](#def-b3-clt-gaussianvector) limit. (b) Here $\sqrt n(\hat p_n - p)
\Rightarrow \mathcal N(0, p(1 - p))$ and $g(p) =
\arcsin\sqrt p$ has $g'(p) = \frac1{2\sqrt{p(1 - p)}}$, so $g'(p)^2\,p(1 - p) = \frac14$: the limit is $\mathcal
N(0, \frac14)$ for every $p \in \intoo01$. On the $\arcsin$ scale the asymptotic $95\%$ error bar is $\pm
\frac{0.98}{\sqrt n}$, known in advance — whereas in [Example 23.9](#ex-b3-clt-confidence) the width involved the unknown $\sigma = \sqrt{p(1-p)}$, to be worst-cased by $\frac12$ or estimated: the transformation *stabilizes* the variance.

**20.** Let $A^* = \{k : \mu(\{k\}) > \nu(\{k\})\}$ and $\Delta_k = \mu(\{k\}) - \nu(\{k\})$, so $\sum_k\Delta_k = 0$. For any $A \subseteq \N$: $\mu(A) -
\nu(A) = \sum_{k\in A}\Delta_k \leq \sum_{k\in
A^*}\Delta_k$, with equality at $A = A^*$; and since the positive and negative parts of $(\Delta_k)$ have equal total mass, $\sum_{A^*}\Delta_k =
\frac12\sum_k\abs{\Delta_k}$. Exchanging $\mu, \nu$ handles the sign: $d_{\mathrm{TV}}(\mu, \nu) =
\frac12\sum_k\abs{\Delta_k}$. Coupling: for any $A$,

$$
\mu(A) - \nu(A) = \E\bigl[\mathbf 1_A(X) - \mathbf
1_A(Y)\bigr] = \E\bigl[(\mathbf 1_A(X) - \mathbf
1_A(Y))\,\mathbf 1_{X\neq Y}\bigr] \leq \P(X \neq Y),
$$

and take the supremum over $A$.

**21.** The two laws charge: $k = 0$: $1 - p$ versus $\eu^{-p}$, with $\eu^{-p} > 1 - p$; $k = 1$: $p$ versus $p\,\eu^{-p} < p$; $k \geq 2$: $0$ versus the Poisson remainder $1 - \eu^{-p} - p\eu^{-p} \geq 0$. Hence

$$
d_{\mathrm{TV}} = \tfrac12\bigl[(\eu^{-p} - 1 + p) + (p -
p\eu^{-p}) + (1 - \eu^{-p} - p\eu^{-p})\bigr] =
\tfrac12\bigl(2p - 2p\eu^{-p}\bigr) = p(1 - \eu^{-p}),
$$

and $1 - \eu^{-p} \leq p$ gives the bound $p^2$.

**22.** Write $H_{i-1} = W_i + X_i$ and $H_i = W_i +
Y_i$ with $W_i = \sum_{j<i}Y_j + \sum_{j>i}X_j$, [independent](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence) of the pair $(X_i, Y_i)$ (coalitions). For $A
\subseteq \N$, conditioning on the countably many values by [independence](https://one-course.com/books/math/5/en/chapter/22-probability-foundations-and-the-law-of-large-numbers#def-b3-probability-independence),

$$
\P(H_{i-1} \in A) = \sum_{k\geq0}\P(X_i = k)\,\P(W_i + k
\in A),
$$

and likewise for $H_i$ with $Y_i$. Subtracting, with $c_k
= \P(W_i + k \in A) \in \intcc01$ and $\Delta_k = \P(X_i =
k) - \P(Y_i = k)$ of zero sum:

$$
\abs{\P(H_{i-1} \in A) - \P(H_i \in A)} =
\Bigl|\sum_k\Delta_k\bigl(c_k - \tfrac12\bigr)\Bigr| \leq
\tfrac12\sum_k\abs{\Delta_k} =
d_{\mathrm{TV}}\bigl(\mathcal B(1, p_i), \mathcal
P(p_i)\bigr) .
$$

Telescoping from $H_0 = S$ to $H_n = \sum_iY_i \sim
\mathcal P(\lambda)$ ([Exercise 23.1](#exo-b3-clt-1), iterated) and using question 21:

$$
\abs{\P(S \in A) - \P(\mathcal P(\lambda) \in A)} \leq
\sum_{i=1}^np_i\bigl(1 - \eu^{-p_i}\bigr) \leq
\sum_{i=1}^np_i^2
\qquad\text{for every } A :
$$

Le Cam’s inequality. (Question 20’s coupling bound gives an alternative route: couple each pair on one uniform variable so that $\P(X_i \neq Y_i) \leq p_i^2$ and bound $\P(S \neq \sum Y_i)$; swapping needs no construction at all.)

**23.** (a) With $p_i = \frac\lambda n$: $d_{\mathrm{TV}}(\text{law of }S, \mathcal P(\lambda))
\leq \frac{\lambda^2}n$. This sharpens [Exercise 23.5](#exo-b3-clt-5) three times over: an explicit error at every finite $n$, uniformity over all events $A$ at once (not one interval at a time), and no need for equal $p_i$ — only $\sum_ip_i^2$ small, e.g. $\sum p_i^2 \leq
\lambda\max_ip_i$: *many rare events, none dominant*. (b) Here $n = 500$, $p_i = \frac1{500}$, $\lambda = 1$: the Poisson model errs by at most $500\cdot\frac1{500^2} = 0.002$ on every event; in particular, taking $A = \{0\}$,

$$
\P(\text{no letter astray}) = \Bigl(1 -
\frac1{500}\Bigr)^{500},
\qquad
\Bigl|\P(\text{no letter astray}) - \eu^{-1}\Bigr| \leq
0.002,
$$

so the answer is $\eu^{-1} \approx 0.368$ up to a guaranteed $0.002$ (the true discrepancy is about $4\cdot10^{-4}$). (c) The problem closes on one method with two regimes. When $n$ comparable contributions each carry variance $\frac1n$, matching *two* moments against the [Gaussian](#def-b3-clt-gaussianvector) makes the swap errors $o(\frac1n)$ each: sums go [Gaussian](#def-b3-clt-gaussianvector) — with Taylor as the local comparison tool. When $n$ contributions are indicators of probability $p_i$, matching the *mean* against a Poisson atom makes each swap cost $p_i^2$: counts of rare events go Poisson — with total variation as the exact local comparison. Same hybrids, same telescope, different local estimate: replacement is a strategy, not a theorem, and the [Gaussian](#def-b3-clt-gaussianvector) and Poisson limits are its two oldest dividends.

**24.** The CLT gives $\sqrt n(\bar X_n - \mu)
\Rightarrow \mathcal N(0, \sigma^2)$, and $g(x) = \ln x$ is differentiable at $\mu > 0$ with $g'(\mu) = \frac1\mu$: the delta method (Part VI) yields $\sqrt n(\ln\bar X_n - \ln\mu)
\Rightarrow \mathcal N(0, \sigma^2/\mu^2)$. Unwinding the interval $\abs{\ln\bar X_n - \ln\mu} \leq
\frac{1.96\,\sigma}{\mu\sqrt n}$ by exponentiation:

$$
\mu \in \bar X_n\cdot
\eu^{\pm1.96\,\sigma/(\mu\sqrt n)}
\qquad\text{with asymptotic probability } 95\%
$$

(in practice $\sigma/\mu$ is replaced by its empirical version, Slutsky as in [Exercise 23.8](#exo-b3-clt-8)). The multiplicative interval is the natural one when the data are positive with errors proportional to their size — incomes, concentrations, half-lives: quantities that live on a log scale, where symmetric additive intervals could even cross zero.

**25.** $\E[(X - p)^3] = (1-p)^3p + (-p)^3(1 - p) =
p(1-p)\bigl[(1-p)^2 - p^2\bigr] = p(1-p)(1 - 2p)$. In the swapping analysis (Part IV), the leading error term after matching two moments carries the *signed* third moment: for $p < \frac12$ it is positive (the law leans right: rare large excursions above the mean), and the normal approximation systematically misplaces mass — underestimating the short left tail and overestimating the right — with an error of order $n^{-1/2}$; at $p =
\frac12$ the third moment vanishes, the Bernoulli matches the [Gaussian](#def-b3-clt-gaussianvector) to third order, and the rate improves (Part IV’s matched-moment question). Numerically: $\P(S = 0) =
0.9^{20} = 0.1216$, while the [Gaussian](#def-b3-clt-gaussianvector) $\mathcal N(2, 1.8)$ gives $\Phi\bigl(\frac{0.5 -
2}{\sqrt{1.8}}\bigr) = \Phi(-1.118) \approx 0.132$: the normal curve, ignorant of the wall at $0$ and of the rightward skew, puts too much mass at the bottom — the predicted sign of the error, visible at $n = 20$.