45.05.03 · mathematical-statistics / 05-empirical-processes-nonparametric

Donsker Classes and the Empirical-Process Weak Limit

shipped3 tiersLean: none

Anchor (Master): van der Vaart 1998 Asymptotic Statistics (Cambridge) Ch. 18-19; van der Vaart & Wellner 1996 Weak Convergence and Empirical Processes (Springer) Part 2 (§2.1 the Donsker theorem, §2.3 maximal inequalities, §2.5 the bracketing and uniform-entropy central limit theorems); Dudley 1999 Uniform Central Limit Theorems (Cambridge) Ch. 3-6

Intuition Beginner

You have already met the empirical distribution: build a staircase from your data and, as the sample grows, the staircase hugs the true curve everywhere at once. That is a law-of-large-numbers statement — the error goes to zero. This unit asks the next question. The error is not just shrinking; it has a shape. If you magnify the gap between the staircase and the true curve by the right amount, what does the magnified error look like?

The answer is a fixed kind of randomness. Blow up the estimation error by the square root of the sample size and the error curve stops collapsing to zero and instead settles into a single, universal random wiggle — the same shape no matter what the original data distribution was, after a simple rescaling. It is the continuous random curve called a Brownian bridge: a jittery path pinned to zero at both ends. So the whole error function, magnified, converges to one fixed random object you can study once and reuse.

The deeper move is that nothing forces you to track only the staircase. You can index your estimate by any menu of questions — any family of summaries of the data — and ask whether the whole magnified error, viewed across that entire menu at once, settles into a fixed random shape. When it does, we call the menu a Donsker class. Glivenko-Cantelli was the promise that errors vanish over a menu; Donsker is the sharper promise that the magnified errors converge to a single Gaussian wiggle over the whole menu together.

The one-sentence takeaway: magnify the estimation error of the empirical distribution by the square root of the sample size, and over a well-behaved menu of questions the whole magnified error converges to one universal Gaussian random curve.

Visual Beginner

Picture the difference between the data staircase and the true curve — the error function — for one sample. It is small and ragged. Now magnify it vertically by the square root of the sample size. The magnified error stops being small; it becomes a wiggly curve of a stable typical size that does not shrink as you collect more data.

Sample size	Raw error (staircase minus true curve)	Error magnified by square root of sample size
Small	Big and blocky	A coarse wiggle, very rough
Medium	Smaller, finer	A finer wiggle of about the same overall height
Large	Tiny	A smooth-looking jittery curve pinned to zero at both ends
Growing without bound	Shrinks to zero	Settles into one fixed random shape: the Brownian bridge

The shape to watch is pinned at both ends. The magnified error is forced to be zero at the far left (no data is below the smallest possible value) and zero at the far right (all the data is below the largest), so the limiting wiggle is tied down at both ends — that is what makes it a bridge rather than a free random walk. Between the ends it is free to wander, and the typical amount it wanders is what goodness-of-fit tests like Kolmogorov-Smirnov measure.

Worked example Beginner

We compute the typical size of the magnified error at a single point, and check it does not shrink with more data. Take data drawn uniformly from the interval $0$ to $1$ , so the true curve is the straight line $F (t) = t$ . Fix the point $t = 0.5$ .

Step 1. Count below the point. The staircase value at $t = 0.5$ is the fraction of the data at or below $0.5$ . Each data point is below $0.5$ with probability $F (0.5) = 0.5$ , like a coin flip. With $n$ data points, the count below $0.5$ behaves like the number of heads in $n$ fair coin flips.

Step 2. Find the spread of that count. The number of heads in $n$ fair flips has spread-squared equal to $n$ times $0.5$ times $0.5$ , which is $n /4$ . Dividing the count by $n$ to make it a fraction divides the spread-squared by $n$ times $n$ , giving spread-squared $1/ (4 n)$ for the staircase value.

Step 3. Magnify by the square root of $n$ . Multiplying the error by the square root of $n$ multiplies its spread-squared by $n$ . So the magnified error at $0.5$ has spread-squared equal to $n$ times $1/ (4 n)$ , which is exactly $1/4$ — no $n$ left in it.

Step 4. Compare to the Brownian bridge. The Brownian bridge at the midpoint $0.5$ has spread-squared $0.5$ times $(1 - 0.5)$ , which is $0.25$ , the same $1/4$ . The numbers match.

Step 5. What this tells us. The raw error shrinks, but magnified by the square root of $n$ it has a spread of one half at the midpoint for every sample size. That stable size is the signature of a limit that does not vanish, and its value $0.25 = t (1 - t)$ is exactly the variance of the Brownian bridge. The magnified error is converging to that bridge.

Check your understanding Beginner

Exercise (easy, multiple choice).

To keep the estimation error of the empirical distribution from collapsing to zero, so it converges to a fixed random shape, by how much should you magnify it?

A. By the sample size itself B. By the square root of the sample size C. By the square of the sample size D. Not at all; the raw error already converges to a fixed shape

Hint

The error's typical size shrinks like one over the square root of the sample size, so the magnifier must cancel exactly that.

Answer

B. By the square root of the sample size. The raw error has typical size of order one over the square root of $n$ , so multiplying by the square root of $n$ leaves a stable, non-vanishing wiggle. Feedback-correct: this is the same square-root scaling as the central limit theorem, applied to the whole error curve at once. Feedback-wrong: A and C over-magnify and the curve blows up to noise; D is the Glivenko-Cantelli statement that the raw error vanishes, which is exactly why some magnification is needed to see a limit.

Formal definition Intermediate+

Throughout, $X_{1}, X_{2}, \dots$ are independent and identically distributed random elements of a measurable space $(X, A)$ with common law $P$ , and $F$ is a class of measurable functions $f : X \to R$ . The empirical measure $P_{n} = n^{- 1} \sum_{k = 1}^{n} δ_{X_{k}}$ , the notation $P_{n} f$ , $P f$ , and the supremum norm $∥ \cdot ∥_{F}$ are those fixed in 45.05.02. The development follows van der Vaart ^{[van der Vaart 1998]} Ch. 18-19 and van der Vaart-Wellner ^{[van der Vaart Wellner 1996]} §1.5, §2.1.

Definition (the empirical process indexed by a class). The empirical process is the map

G_{n} : F \to R, G_{n} f = n (P_{n} f - P f) = \frac{1}{n} k = 1 \sum n (f (X_{k}) - P f) .

For each fixed $f$ with $P f^{2} < \infty$ the ordinary central limit theorem gives $G_{n} f ⇝ N (0, Var_{P} f)$ , and for a finite tuple $(f_{1}, \dots, f_{m})$ the multivariate central limit theorem gives a centred Gaussian limit with covariance $P f_{i} f_{j} - P f_{i} P f_{j}$ . Viewed across all of $F$ at once, $G_{n}$ is a random function on $F$ ; when $sup_{f} ∣ G_{n} f ∣ < \infty$ it is a random element of the space $ℓ^{\infty} (F)$ of bounded real functions on $F$ , normed by $∥ z ∥_{F} = sup_{f \in F} ∣ z (f) ∣$ .

Definition ( $P$ -Brownian bridge / limit process). The $P$ -Brownian bridge indexed by $F$ is the centred Gaussian process $G = (G f)_{f \in F}$ with covariance

Cov (G f, G g) = P f g - P f P g, f, g \in F .

A version that is a tight Borel-measurable element of $ℓ^{\infty} (F)$ exists precisely when $(F, ρ_{P})$ is totally bounded and the sample paths $f \mapsto G f$ are uniformly $ρ_{P}$ -continuous, where $ρ_{P} (f, g) = (Var_{P} (f - g))^{1/2}$ is the intrinsic (or variance) semimetric. For $X = R$ and $F = {1_{(- \infty, t]}}$ this $G$ is the classical $F$ -Brownian bridge with covariance $F (s \land t) - F (s) F (t)$ .

Definition (weak convergence in $ℓ^{\infty} (F)$ ). Because $sup_{f} ∣ G_{n} f ∣$ need not be Borel measurable, weak convergence is taken in the Hoffmann-Jorgensen sense ^{[Hoffmann-Jorgensen 1991]}: a sequence of maps $Z_{n}$ into $ℓ^{\infty} (F)$ converges weakly to a tight Borel-measurable limit $Z$ , written $Z_{n} ⇝ Z$ , if $E^{*} h (Z_{n}) \to E h (Z)$ for every bounded continuous $h : ℓ^{\infty} (F) \to R$ , where $E^{*}$ is outer expectation. The limit being tight and Borel is part of the definition.

Definition ( $P$ -Donsker class). The class $F$ is $P$ -Donsker if

G_{n} ⇝ G in ℓ^{\infty} (F),

with $G$ the tight $P$ -Brownian bridge of the second definition. Necessarily then $F$ has $P$ -square-integrable envelope behaviour making $G_{n}$ asymptotically bounded, $(F, ρ_{P})$ is totally bounded, and (by the continuous-mapping theorem applied to $z \mapsto ∥ z ∥_{F}$ ) the suprema $∥ G_{n} ∥_{F}$ converge in law to $∥ G ∥_{F}$ .

Definition (asymptotic equicontinuity). The empirical process is asymptotically $ρ_{P}$ -equicontinuous if for every $ε > 0$

δ ↓ 0 lim n \to \infty lim sup P^{*} (ρ_{P} (f, g) < δ sup G_{n} f - G_{n} g > ε) = 0,

where $P^{*}$ is outer probability. This is the function-class analogue of the modulus-of-continuity tightness criterion on $C [0, 1]$ used for the one-dimensional invariance principle 37.03.03; it is the path-space content of the empirical CLT, separate from the finite-dimensional (multivariate CLT) content.

Key theorem with proof Intermediate+

Theorem (the abstract empirical central limit theorem). Let $F$ be a class of measurable functions with $(F, ρ_{P})$ totally bounded. Then $F$ is $P$ -Donsker if and only if the empirical process $G_{n}$ is asymptotically $ρ_{P}$ -equicontinuous. In that case $G_{n} ⇝ G$ , the tight $P$ -Brownian bridge with covariance $P f g - P f P g$ .

Proof. The argument is the function-class version of the Prokhorov template — finite-dimensional convergence plus tightness — adapted to the non-separable space $ℓ^{\infty} (F)$ following ^{[van der Vaart Wellner 1996]} §1.5. Weak convergence to a tight limit in $ℓ^{\infty} (F)$ holds if and only if (i) all finite-dimensional marginals converge and (ii) the sequence is asymptotically tight, and for a totally bounded index semimetric asymptotic tightness is equivalent to asymptotic equicontinuity. We supply both directions.

Finite-dimensional marginals. For any finite tuple $f_{1}, \dots, f_{m} \in F$ with $P f_{i}^{2} < \infty$ , the vector $(G_{n} f_{1}, \dots, G_{n} f_{m})$ is $n^{- 1/2} \sum_{k} (f_{i} (X_{k}) - P f_{i})_{i}$ , a normalised sum of i.i.d. mean-zero random vectors with covariance $Σ_{ij} = P f_{i} f_{j} - P f_{i} P f_{j}$ . The multivariate central limit theorem gives convergence to $N (0, Σ)$ , which is exactly the marginal $(G f_{1}, \dots, G f_{m})$ of the $P$ -Brownian bridge. So (i) holds for every class with square-integrable elements, with no condition on the size of $F$ .

Equicontinuity equals tightness. Suppose $G_{n}$ is asymptotically $ρ_{P}$ -equicontinuous. Fix $η, ε > 0$ and choose $δ$ so that $lim sup_{n} P^{*} (sup_{ρ_{P} (f, g) < δ} ∣ G_{n} f - G_{n} g ∣ > ε) < η$ . Total boundedness of $(F, ρ_{P})$ furnishes a finite $δ$ -net $f_{1}, \dots, f_{p}$ . Every $f$ lies within $ρ_{P}$ -distance $δ$ of some net point $f_{j (f)}$ , so

∥ G_{n} ∥_{F} \leq 1 \leq j \leq p max ∣ G_{n} f_{j} ∣ + ρ_{P} (f, g) < δ sup ∣ G_{n} f - G_{n} g ∣.

The first term is a maximum of $p$ asymptotically Gaussian coordinates, hence asymptotically tight in $R$ ; the second is below $ε$ with limiting probability at least $1 - η$ . This is precisely the asymptotic-tightness criterion: the finitely many net coordinates are tight and the oscillation over $δ$ -balls is uniformly small, so ${G_{n}}$ is asymptotically tight in $ℓ^{\infty} (F)$ . Conversely, a tight limit $G$ has $ρ_{P}$ -uniformly-continuous sample paths (a tight Borel law on $ℓ^{\infty} (F)$ concentrates on the separable subspace $U C (F, ρ_{P})$ ), and asymptotic tightness of $G_{n}$ forces the equicontinuity modulus to vanish in the stated double limit.

Identification and conclusion. Asymptotic tightness plus marginal convergence give, along every subsequence, a weakly convergent sub-subsequence whose limit has the Gaussian marginals computed above; since a tight Borel law on $ℓ^{\infty} (F)$ is determined by its finite-dimensional marginals, every subsequential limit is the same $P$ -Brownian bridge $G$ , so the whole sequence converges: $G_{n} ⇝ G$ . The bridge is supported on the uniformly $ρ_{P}$ -continuous functions, with $ρ_{P}$ totally bounded, exactly matching the limit demanded by the Donsker definition. $□$

Bridge. This equicontinuity criterion builds toward the entropy sufficient conditions of 45.05.04, where a bound on the bracketing or uniform covering numbers of $F$ is converted, by a chaining maximal inequality, into the asymptotic equicontinuity that the theorem requires; it appears again in the functional delta method 45.05.06, where a statistic that is a smooth functional of $P_{n}$ inherits a Gaussian limit by composing the Donsker limit $G_{n} ⇝ G$ with a Hadamard derivative. The foundational reason the abstract CLT splits this way is that weak convergence in $ℓ^{\infty} (F)$ factors into a finite-dimensional part, supplied for free by the multivariate central limit theorem, and a uniformity part, which is the entire difficulty and is exactly governed by the metric entropy of $F$ in the variance semimetric. This is exactly the function-class generalisation of the one-dimensional invariance principle 37.03.03: there the index set is the time interval $[0, 1]$ , the semimetric is $ρ (s, t) = ∣ t - s ∣^{1/2}$ , the limit is the Brownian bridge on $[0, 1]$ , and tightness is the modulus-of-continuity criterion; here the index set is an abstract class $F$ , the semimetric is $ρ_{P}$ , the limit is the $P$ -Brownian bridge, and tightness is asymptotic $ρ_{P}$ -equicontinuity.

Putting these together, the Donsker property is to Glivenko-Cantelli 45.05.02 what the central limit theorem is to the law of large numbers, and the central insight is that a single condition on the size of $F$ — small enough metric entropy — simultaneously delivers the uniform law (Glivenko-Cantelli) and its Gaussian fluctuation (Donsker); the bridge is the passage from convergence of one rescaled coordinate to convergence of the whole rescaled error process indexed by $F$ .

Exercises Intermediate+

Exercise 1 (easy, symbolic).

Show that the classical empirical-process limit is the special case of the abstract CLT for the half-line indicators. Take $X = R$ , $F = {1_{(- \infty, t]} : t \in R}$ , and compute the limit covariance $Cov (G f_{s}, G f_{t})$ for $f_{t} = 1_{(- \infty, t]}$ . Identify the limit process.

Hint

Use $P f_{s} f_{t} = P ({X \leq s} \cap {X \leq t}) = F (s \land t)$ and $P f_{t} = F (t)$ in the covariance $P f g - P f P g$ .

Answer

Here $f_{s} f_{t} = 1_{(- \infty, s]} 1_{(- \infty, t]} = 1_{(- \infty, s \land t]}$ , so $P f_{s} f_{t} = F (s \land t)$ and $P f_{t} = F (t)$ . The covariance is $F (s \land t) - F (s) F (t)$ . The limit process $G f_{t} =: G (t)$ is the centred Gaussian process with this covariance, the $F$ -Brownian bridge; for $F$ continuous, the time-change $t \mapsto F (t)$ turns it into the standard Brownian bridge $B^{0}$ on $[0, 1]$ with covariance $u \land v - uv$ . Thus $n (F_{n} - F) ⇝$ the $F$ -Brownian bridge, recovering the classical empirical-process Donsker theorem 37.03.03 as the half-line-indicator case.

Exercise 2 (easy, true-false).

Every $P$ -Donsker class is $P$ -Glivenko-Cantelli.

Hint

If $n ∥ P_{n} - P ∥_{F} = ∥ G_{n} ∥_{F}$ converges in law to a finite random variable, what does dividing by $n$ do?

Answer

True. If $F$ is $P$ -Donsker then $∥ G_{n} ∥_{F} = n ∥ P_{n} - P ∥_{F} ⇝ ∥ G ∥_{F}$ by the continuous-mapping theorem applied to the norm, and the limit is a finite (a.s. bounded) random variable. Hence $∥ G_{n} ∥_{F} = O_{P^{*}} (1)$ , so $∥ P_{n} - P ∥_{F} = O_{P^{*}} (n^{- 1/2}) \to 0$ in outer probability. Under the mild measurability/envelope conditions that accompany the Donsker property this upgrades to outer-almost-sure convergence, so $F$ is Glivenko-Cantelli. The converse fails: a class can be GC (errors vanish) without being Donsker (the $n$ -magnified error need not have a tight Gaussian limit).

Exercise 3 (medium, symbolic).

Show that any finite class $F = {f_{1}, \dots, f_{m}}$ with $P f_{j}^{2} < \infty$ for each $j$ is $P$ -Donsker, and identify the limit explicitly.

Hint

On a finite index set $ℓ^{\infty} (F) = R^{m}$ and weak convergence is ordinary convergence in distribution; apply the multivariate CLT.

Answer

For finite $F$ the space $ℓ^{\infty} (F)$ is just $R^{m}$ with the max norm, on which weak convergence coincides with convergence in distribution of random vectors, and any limit is automatically tight and Borel. The vector $(G_{n} f_{1}, \dots, G_{n} f_{m}) = n^{- 1/2} \sum_{k} (f_{j} (X_{k}) - P f_{j})_{j}$ is a normalised i.i.d. sum, so by the multivariate central limit theorem [37.03.03 prerequisite chain] it converges to $N (0, Σ)$ with $Σ_{ij} = P f_{i} f_{j} - P f_{i} P f_{j}$ . This $N (0, Σ)$ is exactly the $P$ -Brownian bridge restricted to the finite index, so $F$ is $P$ -Donsker. Equicontinuity is vacuous: with finitely many points the variance semimetric is discrete, so for small enough $δ$ the only pairs with $ρ_{P} (f, g) < δ$ are $f = g$ .

Exercise 4 (medium, symbolic).

Let $F$ be $P$ -Donsker and let $ϕ : R \to R$ be Lipschitz with constant $L$ and $ϕ (0) = 0$ . Using only the asymptotic-equicontinuity criterion, show $ϕ \circ F = {ϕ \circ f : f \in F}$ has empirical process that is asymptotically equicontinuous in the semimetric inherited from $F$ (assume square-integrable envelopes throughout). What does this say about preservation of the Donsker property?

Hint

Bound $ρ_{P} (ϕ \circ f, ϕ \circ g)$ by $L ρ_{P} (f, g)$ using $∣ ϕ (u) - ϕ (v) ∣ \leq L ∣ u - v ∣$ inside the variance.

Answer

By Lipschitzness, $∣ ϕ (f (x)) - ϕ (g (x)) ∣ \leq L ∣ f (x) - g (x) ∣$ pointwise, so $Var_{P} (ϕ \circ f - ϕ \circ g) \leq P (ϕ \circ f - ϕ \circ g)^{2} \leq L^{2} P (f - g)^{2}$ , giving $ρ_{P} (ϕ \circ f, ϕ \circ g) \leq L ρ_{P} (f, g)$ . Hence a $ρ_{P}$ -ball of radius $δ / L$ in $F$ maps into a ball of radius $δ$ around $ϕ \circ f$ , and total boundedness of $(F, ρ_{P})$ transfers to $(ϕ \circ F, ρ_{P})$ . For the oscillation, $∣ G_{n} (ϕ \circ f) - G_{n} (ϕ \circ g) ∣$ is controlled by the increments of $G_{n}$ over pairs that are $ρ_{P}$ -close in $F$ after the $L$ -contraction, so the equicontinuity modulus of $ϕ \circ F$ is dominated by that of $F$ rescaled by $L$ , and vanishes in the double limit. Therefore $ϕ \circ F$ inherits asymptotic equicontinuity, and with the marginal CLT it is $P$ -Donsker: the Donsker property is stable under composition with a fixed Lipschitz map, one of the preservation theorems that make the class of Donsker classes a usable calculus.

Exercise 5 (medium, symbolic).

Suppose $F$ is $P$ -Donsker and $g_{n}, g \in F$ with $ρ_{P} (g_{n}, g) \to 0$ . Show $G_{n} g_{n} - G_{n} g \to 0$ in outer probability, and deduce $G_{n} g_{n} ⇝ G g$ .

Hint

For $ρ_{P} (g_{n}, g) < δ$ the difference is below the equicontinuity modulus; combine with $G_{n} g ⇝ G g$ .

Answer

Fix $ε > 0$ . Since $ρ_{P} (g_{n}, g) \to 0$ , for any $δ$ there is $N$ with $ρ_{P} (g_{n}, g) < δ$ for $n \geq N$ , so for such $n$

P^{*} (∣ G_{n} g_{n} - G_{n} g ∣ > ε) \leq P^{*} (ρ_{P} (f, f^{'}) < δ sup ∣ G_{n} f - G_{n} f^{'} ∣ > ε) .

Taking $lim sup_{n}$ and then $δ ↓ 0$ , the right side vanishes by asymptotic equicontinuity (a consequence of $F$ being Donsker), so $G_{n} g_{n} - G_{n} g \to 0$ in outer probability. The single coordinate $G_{n} g ⇝ G g = N (0, Var_{P} g)$ by the marginal CLT, and the converging-together (Slutsky) lemma gives $G_{n} g_{n} ⇝ G g$ . This "equicontinuity along a shrinking sequence" is the workhorse behind plug-in linearisation: it lets an estimated index $g_{n}$ be replaced by its limit $g$ inside the empirical process at no asymptotic cost.

Exercise 6 (medium, numeric).

For the standard Brownian bridge limit $B^{0}$ of the uniform empirical process, the Kolmogorov-Smirnov limit law is $P (sup_{t} ∣ B_{t}^{0} ∣ \leq x) = 1 - 2 \sum_{j \geq 1} (- 1)^{j - 1} e^{- 2 j^{2} x^{2}}$ . Using only the leading term, estimate the $0.95$ quantile $x$ solving $2 e^{- 2 x^{2}} = 0.05$ . Give $x$ to two decimal places.

Hint

Solve $2 e^{- 2 x^{2}} = 0.05$ for $x$ : take logs, $2 x^{2} = lo g (2/0.05)$ .

Answer

$1.36$ . From $2 e^{- 2 x^{2}} = 0.05$ , $e^{- 2 x^{2}} = 0.025$ , so $2 x^{2} = lo g (1/0.025) = lo g 40 \approx 3.689$ , giving $x^{2} \approx 1.844$ and $x \approx 1.358$ . The exact Kolmogorov $0.95$ quantile (summing the full series) is $1.358$ , so the leading term already gives the standard critical value $1.36$ used in the Kolmogorov-Smirnov test. The whole test statistic is the continuous functional $z \mapsto ∥ z ∥_{\infty}$ applied to the Donsker limit, illustrating how a single weak limit on $ℓ^{\infty} (F)$ delivers a usable inferential threshold.

Exercise 7 (hard, symbolic).

Prove the finite-dimensional plus asymptotic-equicontinuity implies Donsker direction in the special case of a totally bounded class, by giving the explicit net argument: assuming asymptotic equicontinuity and total boundedness, show $∥ G_{n} ∥_{F}$ is asymptotically tight in $R$ and that $G_{n}$ is asymptotically tight in $ℓ^{\infty} (F)$ .

Hint

Cover $F$ by a finite $δ$ -net; split the supremum into a max over the net plus the oscillation over $δ$ -balls.

Answer

Fix $ε, η > 0$ . By asymptotic equicontinuity pick $δ > 0$ with $lim sup_{n} P^{*} (ω_{n} (δ) > ε) < η /2$ , where $ω_{n} (δ) = sup_{ρ_{P} (f, g) < δ} ∣ G_{n} f - G_{n} g ∣$ . By total boundedness take a finite $δ$ -net ${f_{1}, \dots, f_{p}}$ . For each $f$ pick $f_{j (f)}$ with $ρ_{P} (f, f_{j (f)}) < δ$ ; then $∣ G_{n} f ∣ \leq max_{j} ∣ G_{n} f_{j} ∣ + ω_{n} (δ)$ , so $∥ G_{n} ∥_{F} \leq max_{j} ∣ G_{n} f_{j} ∣ + ω_{n} (δ)$ . The finite max converges in law (multivariate CLT) to $max_{j} ∣ G f_{j} ∣$ , hence is asymptotically tight: choose $M$ with $lim sup_{n} P (max_{j} ∣ G_{n} f_{j} ∣ > M) < η /2$ . Combining, $lim sup_{n} P^{*} (∥ G_{n} ∥_{F} > M + ε) < η$ , so $∥ G_{n} ∥_{F}$ is asymptotically tight in $R$ . For tightness in $ℓ^{\infty} (F)$ , the asymptotic-tightness criterion in $ℓ^{\infty}$ requires for each $ε, η$ a *finite* partition $F = ⨆_{i} F_{i}$ with $lim sup_{n} P^{*} (max_{i} sup_{f, g \in F_{i}} ∣ G_{n} f - G_{n} g ∣ > ε) < η$ ; the partition into $δ$ -net cells supplies exactly this, since within a cell $ρ_{P} (f, g) < 2 δ$ and the oscillation is controlled by $ω_{n} (2 δ)$ . With the marginal convergence already in hand, the abstract weak-convergence theorem in $ℓ^{\infty} (F)$ delivers $G_{n} ⇝ G$ .

Exercise 8 (hard, symbolic).

Let $F$ be $P$ -Donsker with $∥ G_{n} ∥_{F} ⇝ ∥ G ∥_{F}$ . Let $T : ℓ^{\infty} (F) \to R$ be Hadamard-differentiable at $P$ (regarded inside $ℓ^{\infty} (F)$ ) tangentially to the support of $G$ , with derivative $T_{P}^{'}$ . State and justify the resulting limit law for $n (T (P_{n}) - T (P))$ , and explain why the Donsker property of $F$ is the hypothesis that makes the functional delta method apply.

Hint

Write $n (T (P_{n}) - T (P))$ via $n (P_{n} - P) = G_{n} ⇝ G$ and apply the functional delta method (Hadamard differentiability plus weak convergence).

Answer

Since $n (P_{n} - P) = G_{n} ⇝ G$ in $ℓ^{\infty} (F)$ (the Donsker hypothesis) and $T$ is Hadamard-differentiable at $P$ tangentially to a set containing the support of $G$ , the functional delta method gives

n (T (P_{n}) - T (P)) = T_{P}^{'} (G_{n}) + o_{P^{*}} (1) ⇝ T_{P}^{'} (G) .

Because $T_{P}^{'}$ is a continuous linear map and $G$ is Gaussian, $T_{P}^{'} (G)$ is centred Gaussian (a real Gaussian when $T$ is real-valued), with variance $Var (T_{P}^{'} G)$ read off from the bridge covariance $P f g - P f P g$ . The Donsker property is exactly the enabling hypothesis: the delta method requires its input sequence $n (P_{n} - P)$ to converge weakly to a tight limit in the same space where $T$ is differentiated, and " $F$ is $P$ -Donsker" is precisely the statement that this convergence holds in $ℓ^{\infty} (F)$ . Statistics expressible as Hadamard-differentiable functionals of the empirical measure over a Donsker class — quantiles, trimmed means, the Cramér-von Mises and Anderson-Darling statistics, Z- and M-estimators — therefore acquire $n$ -Gaussian limits in one stroke 45.05.06.

Advanced results Master

Theorem 1 (Donsker via bracketing entropy). If the bracketing integral is finite,

J_{[]} (\infty, F, L^{2} (P)) = \int_{0}^{\infty} lo g N_{[]} (ε, F, L^{2} (P)) d ε < \infty,

then $F$ is $P$ -Donsker ^{[van der Vaart Wellner 1996]}. The mechanism is a chaining maximal inequality: bracketing the class at a geometric sequence of scales and summing the increments controls $E^{*} sup_{ρ_{P} (f, g) < δ} ∣ G_{n} f - G_{n} g ∣$ by a tail of the bracketing integral, which is the asymptotic equicontinuity the Key theorem demands. The integrand $lo g N_{[]}$ rather than $lo g N_{[]}$ is the quantitative gain of the central limit theorem over the law of large numbers: Glivenko-Cantelli needs only finiteness of $N_{[]}$ at each scale 45.05.02, the Donsker property needs the entropy to grow slowly enough that its square root integrates. The detailed entropy bookkeeping is 45.05.04.

Theorem 2 (Donsker via uniform entropy). If $F$ has a square-integrable envelope $F$ ( $P F^{2} < \infty$ ) and finite uniform-entropy integral

J (\infty, F) = \int_{0}^{\infty} Q sup lo g N (ε ∥ F ∥_{Q, 2}, F, L^{2} (Q)) d ε < \infty,

the supremum over finitely supported probability measures $Q$ , together with a mild measurability (image-admissibility) condition, then $F$ is $P$ -Donsker ^{[Dudley 1978]}. Every VC class of functions with square-integrable envelope satisfies this bound — by the polynomial discrimination of Sauer's lemma 45.05.02, $sup_{Q} N (ε ∥ F ∥_{Q, 2}, F, L^{2} (Q)) \leq C ε^{- W}$ for a constant depending on the VC index, whose square-rooted logarithm $W lo g (1/ ε)$ integrates near $0$ . So finite-VC plus a square-integrable envelope implies $P$ -Donsker for every $P$ , the empirical-process analogue of the distribution-free VC Glivenko-Cantelli theorem.

Theorem 3 (preservation calculus for Donsker classes). The Donsker property is stable under the operations of plug-in statistics ^{[van der Vaart 1998]}. If $F$ and $G$ are $P$ -Donsker with square-integrable envelopes, then $F \cup G$ , $F + G = {f + g}$ , the pointwise minima and maxima $F \land G$ , $F \lor G$ , and the image $ϕ \circ (F, G)$ under a fixed Lipschitz $ϕ$ with $ϕ (0)$ controlled are again $P$ -Donsker; products $F \cdot G$ are Donsker when both classes are uniformly bounded. The mechanism is again the variance-semimetric inequality of Exercise 4: each operation is a contraction or finite combination in $ρ_{P}$ , so it preserves both total boundedness and the chaining-controlled equicontinuity. This calculus is what turns the abstract CLT into a tool: complicated classes are assembled from VC or bracketing primitives and inherit the Donsker property by closure.

Theorem 4 (multiplier and bootstrap central limit theorems). If $F$ is $P$ -Donsker, then the multiplier empirical process $G_{n}^{ξ} f = n^{- 1/2} \sum_{k} ξ_{k} (f (X_{k}) - P_{n} f)$ , with $ξ_{k}$ i.i.d. mean-zero variance-one independent of the data, converges weakly conditionally on the data to the same $P$ -Brownian bridge $G$ , in outer probability ^{[van der Vaart Wellner 1996]}. The empirical (Efron) bootstrap, the special case of exchangeable multinomial weights, inherits this: the bootstrap empirical process $n (P_{n}^{*} - P_{n})$ converges conditionally to $G$ . The Donsker property is precisely the condition under which the nonparametric bootstrap is consistent for the law of the empirical process, so confidence bands and goodness-of-fit thresholds can be calibrated by resampling rather than by computing the bridge's law analytically.

Theorem 5 (functional delta method over a Donsker class). Let $T$ be Hadamard-differentiable at $P$ tangentially to a subspace containing the support of $G$ , with continuous linear derivative $T_{P}^{'}$ . If $F$ is $P$ -Donsker then

n (T (P_{n}) - T (P)) ⇝ T_{P}^{'} (G),

a (possibly infinite-dimensional) centred Gaussian limit ^{[van der Vaart 1998]}. The Donsker convergence $G_{n} ⇝ G$ is the input and Hadamard differentiability the transfer; together they handle quantiles (via the inverse-map derivative), the Cramér-von Mises and Anderson-Darling statistics (via integration functionals), trimmed and Winsorised means, and Z- and M-estimators presented as zeros of estimating equations. The systematic development is 45.05.06.

Synthesis. The empirical process $G_{n} = n (P_{n} - P)$ is the central object of asymptotic statistics, and the question of when it converges to a tight Gaussian limit is the question of when $F$ is Donsker. The foundational reason the abstract CLT holds is that weak convergence in $ℓ^{\infty} (F)$ splits cleanly into a finite-dimensional part — the multivariate central limit theorem, free for any square-integrable class — and a uniformity part that is the whole content and is governed entirely by the metric entropy of $F$ in the $L^{2} (P)$ semimetric. This is exactly the dichotomy that runs through the chapter: bracketing entropy controls a class through a fixed reference measure, uniform entropy controls it combinatorially and distribution-free, and the two integrability conditions $\int lo g N_{[]}$ and $\int sup_{Q} lo g N$ are the Donsker-tier strengthenings of the bare finiteness that sufficed for Glivenko-Cantelli 45.05.02.

The Donsker property generalises the one-dimensional invariance principle 37.03.03 from the time-indexed Brownian bridge on $[0, 1]$ to the class-indexed $P$ -Brownian bridge on $ℓ^{\infty} (F)$ , and it is dual to the law-of-large-numbers Glivenko-Cantelli statement in the precise sense that one rescales the same deviation $P_{n} - P$ by $1$ and by $n$ . Putting these together, the single weak limit $G_{n} ⇝ G$ powers three pillars of inference at once: uniform confidence bands (via the continuous norm functional and the Kolmogorov-Smirnov law), the consistency of the nonparametric bootstrap (via the multiplier CLT), and the $n$ -asymptotics of every smooth functional of the data (via the functional delta method 45.05.06). The central insight is that the same finite-VC or finite-bracketing-entropy condition that makes the empirical measure uniformly consistent makes its fluctuation Gaussian, so Glivenko-Cantelli and Donsker are the qualitative skeleton and the quantitative refinement of one statement about the size of $F$ .

Full proof set Master

The abstract empirical CLT (finite-dimensional convergence plus asymptotic equicontinuity equals Donsker), the half-line-indicator special case, the finite-class case, the Lipschitz-composition preservation, and the functional-delta consequence are proved in the Key theorem and Exercises. The remaining Master claims are recorded here.

Proposition 1 (Donsker implies Glivenko-Cantelli, with the $n$ rate). If $F$ is $P$ -Donsker then $∥ P_{n} - P ∥_{F} = O_{P^{*}} (n^{- 1/2})$ , and in particular $F$ is $P$ -Glivenko-Cantelli in outer probability.

Proof. The norm $z \mapsto ∥ z ∥_{F}$ is $1$ -Lipschitz, hence continuous, on $ℓ^{\infty} (F)$ . By the continuous-mapping theorem for weak convergence in $ℓ^{\infty} (F)$ , $G_{n} ⇝ G$ gives $∥ G_{n} ∥_{F} ⇝ ∥ G ∥_{F}$ , a finite random variable (the bridge has bounded sample paths on the totally bounded $(F, ρ_{P})$ ). A weakly convergent sequence of reals is stochastically bounded, so $∥ G_{n} ∥_{F} = O_{P^{*}} (1)$ , i.e. $n ∥ P_{n} - P ∥_{F} = O_{P^{*}} (1)$ , whence $∥ P_{n} - P ∥_{F} = O_{P^{*}} (n^{- 1/2}) \to 0$ in outer probability. This is the Glivenko-Cantelli property in probability with the sharp $n^{- 1/2}$ rate that the qualitative GC theorem of 45.05.02 leaves unquantified. $□$

Proposition 2 (the chaining bound that yields equicontinuity from bracketing entropy). Suppose $F$ has $L^{2} (P)$ -bracketing numbers with $J_{[]} (δ, F, L^{2} (P)) = \int_{0}^{δ} lo g N_{[]} (ε, F, L^{2} (P)) d ε \to 0$ as $δ ↓ 0$ . Then $E^{*} sup_{ρ_{P} (f, g) < δ} ∣ G_{n} f - G_{n} g ∣ ≲ J_{[]} (δ, F, L^{2} (P)) + n P [2 F 1 {2 F > n a (δ)}]$ for an appropriate envelope $F$ and threshold $a (δ)$ , and $F$ is asymptotically equicontinuous.

Proof sketch. This is the bracketing maximal inequality of van der Vaart-Wellner §2.5 ^{[van der Vaart Wellner 1996]}. Build a nested sequence of bracketing partitions of $F$ at scales $ε_{j} = 2^{- j} δ$ , with $N_{[]} (ε_{j}, F, L^{2} (P))$ brackets at level $j$ . Chain: write $f - g = \sum_{j} (π_{j} f - π_{j - 1} f) - \sum_{j} (π_{j} g - π_{j - 1} g)$ as a telescoping sum of successive bracket approximations $π_{j} f$ . Each link is an increment of $G_{n}$ over a pair at $L^{2} (P)$ -distance $\leq ε_{j - 1}$ ; a Bernstein-type maximal inequality over the $N_{[]} (ε_{j})^{2}$ pairs at level $j$ bounds the contribution of level $j$ by $ε_{j - 1} lo g N_{[]} (ε_{j})$ , up to a remainder from the truncation of large bracket-width contributions (the second term, which vanishes under $P F^{2} < \infty$ as the threshold grows with $n$ ). Summing over $j$ produces the entropy integral $\sum_{j} ε_{j - 1} lo g N_{[]} (ε_{j}) ≲ J_{[]} (δ, \cdot)$ . As $δ ↓ 0$ this tends to $0$ , so the equicontinuity modulus vanishes in the double limit, and the Key theorem upgrades this to $G_{n} ⇝ G$ , proving Theorem 1. The full bookkeeping is deferred to 45.05.04. $□$

Proposition 3 (existence and continuity of the $P$ -Brownian bridge). If $(F, ρ_{P})$ is totally bounded and $\int_{0}^{\infty} lo g N (ε, F, ρ_{P}) d ε < \infty$ , then the centred Gaussian process $G$ with covariance $P f g - P f P g$ has a version with $ρ_{P}$ -uniformly-continuous sample paths, defining a tight Borel-measurable element of $ℓ^{\infty} (F)$ .

Proof. The covariance gives $E (G f - G g)^{2} = Var_{P} (f - g) = ρ_{P} (f, g)^{2}$ , so $G$ is a sub-Gaussian process with respect to the intrinsic semimetric $ρ_{P}$ . Dudley's entropy bound for Gaussian processes gives $E sup_{ρ_{P} (f, g) < δ} ∣ G f - G g ∣ ≲ \int_{0}^{δ} lo g N (ε, F, ρ_{P}) d ε$ , which tends to $0$ with $δ$ under the entropy hypothesis ^{[Dudley 1978]}. Hence $G$ has a uniformly $ρ_{P}$ -continuous modification on the totally bounded space $(F, ρ_{P})$ ; its sample paths lie in the separable space $U C (F, ρ_{P}) \subset ℓ^{\infty} (F)$ , and a Gaussian law concentrated on a separable Banach subspace is tight (Borel measures on separable metric spaces are tight). The covariance determines all finite-dimensional marginals, so this version is the unique tight $P$ -Brownian bridge, the limit object in the Donsker definition. $□$

Proposition 4 (the multiplier inequality). Let $F$ have envelope $F$ with $P F^{2} < \infty$ and let $ξ_{1}, \dots, ξ_{n}$ be i.i.d. mean-zero, independent of $X_{1}, \dots, X_{n}$ , with $∥ ξ_{1} ∥_{2, 1} = \int_{0}^{\infty} P (∣ ξ_{1} ∣ > u) d u < \infty$ . Then for the symmetrised process there are constants with

E^{*} \frac{1}{n} k = 1 \sum n ξ_{k} (f (X_{k}) - P f)_{F} ≲ ∥ ξ_{1} ∥_{2, 1} m \leq n max E^{*} \frac{1}{m} k = 1 \sum m ε_{k} (f (X_{k}) - P f)_{F},

with $ε_{k}$ Rademacher signs, so general multipliers are controlled by Rademacher multipliers.

Proof sketch. Condition on the data and on $∣ ξ_{k} ∣$ , decompose each multiplier into a signed magnitude $ξ_{k} = ε_{k} ∣ ξ_{k} ∣$ with $ε_{k}$ Rademacher, and represent the magnitude through its tail by the layer-cake identity $∣ ξ_{k} ∣ = \int_{0}^{\infty} 1 {∣ ξ_{k} ∣ > u} d u$ . Interchanging the sum and the integral writes the multiplier sum as an average of Rademacher sums over thresholded subsets of indices; a maximal inequality over the partial-sum processes (Montgomery-Smith) bounds each by the largest Rademacher sum, and integrating the tail produces the $L^{2, 1}$ norm $∥ ξ_{1} ∥_{2, 1}$ as the multiplicative constant ^{[van der Vaart Wellner 1996]}. The Rademacher process on the right is, by the symmetrisation of 45.05.02 applied at the process level, comparable to $E^{*} ∥ G_{n} ∥_{F}$ , which is $O (1)$ for a Donsker class. Combined with conditional finite-dimensional convergence (a conditional Lindeberg CLT given the data), this yields the conditional weak convergence $G_{n}^{ξ} ⇝ G$ of Theorem 4 and hence bootstrap consistency. $□$

Connections Master

The empirical distribution and Glivenko-Cantelli classes 45.05.02 is the law-of-large-numbers counterpart of this unit: Glivenko-Cantelli rescales the deviation $P_{n} - P$ by $1$ and asks for uniform convergence to zero, the Donsker property rescales it by $n$ and asks for convergence to a Gaussian bridge, and the same metric-entropy condition controls both — finiteness of $N_{[]}$ at each scale for the GC theorem, finiteness of the square-rooted entropy integral $\int lo g N_{[]}$ for the Donsker theorem. The VC-dimension and Sauer-lemma combinatorics proved there reappear here as the route by which a VC class satisfies the uniform-entropy CLT.

Donsker's invariance principle and the functional CLT 37.03.03 is the one-dimensional prototype this unit abstracts: the time interval $[0, 1]$ becomes the function class $F$ , the metric $∣ t - s ∣^{1/2}$ becomes the variance semimetric $ρ_{P}$ , the modulus-of-continuity tightness criterion becomes asymptotic $ρ_{P}$ -equicontinuity, and the Brownian bridge on $[0, 1]$ becomes the $P$ -Brownian bridge on $ℓ^{\infty} (F)$ ; the classical empirical-process limit $n (F_{n} - F) ⇝$ an $F$ -bridge is recovered as the half-line-indicator class, so this unit does not re-prove the one-dimensional invariance principle but builds the class-indexed version on top of it.

The bracketing and uniform-entropy central limit theorems 45.05.04 supply the verifiable sufficient conditions for the abstract criterion of this unit: where the Key theorem reduces the Donsker property to asymptotic equicontinuity, 45.05.04 converts a bound on covering or bracketing numbers into that equicontinuity through the chaining maximal inequality sketched in Proposition 2, making "is $F$ Donsker?" a computation about the size of $F$ .

The functional delta method 45.05.06 is the principal consumer of the Donsker limit: a statistic that is a Hadamard-differentiable functional of the empirical measure over a Donsker class inherits a $n$ -Gaussian limit by composing $G_{n} ⇝ G$ with the derivative $T_{P}^{'}$ , which is how quantiles, the Cramér-von Mises and Anderson-Darling statistics, trimmed means, and Z- and M-estimators acquire their asymptotic distributions in one stroke.

The Vapnik-Chervonenkis theory of learnability 45.07.03 shares the entropy machinery from the opposite side: the finite-VC condition that makes a class Donsker here (uniform-entropy CLT) is the same finite-VC condition that makes empirical risk minimisation generalise there, so the empirical-process CLT and the uniform-convergence bounds of statistical learning are two readings of one statement about the metric entropy of a hypothesis class, with Glivenko-Cantelli the qualitative version and Donsker the Gaussian refinement.

Historical & philosophical context Master

The Brownian-bridge limit for the empirical distribution function was conjectured by Joseph Doob in 1949 on heuristic grounds — he observed that the finite-dimensional distributions of the normalised empirical process match those of a Brownian bridge and asked whether the whole process converges — and proved by Monroe Donsker in 1952 ^{[Donsker 1952]}, who supplied the tightness argument that justifies passing from the finite-dimensional Gaussian limits to weak convergence of the entire process, and from there to the limiting laws of the Kolmogorov-Smirnov and Cramér-von Mises statistics as continuous functionals of the bridge. This is the one-dimensional empirical-process invariance principle, the immediate sibling of the partial-sum invariance principle 37.03.03.

The abstraction to general index classes is due to Richard Dudley, whose 1978 paper on central limit theorems for empirical measures ^{[Dudley 1978]} introduced the metric-entropy conditions and the class of Donsker classes, and established the empirical CLT for classes of sets and functions of finite uniform entropy. The measurability difficulties — the supremum $sup_{f} ∣ G_{n} f ∣$ over an uncountable class is generically non-Borel — were resolved by the theory of outer expectations and weak convergence of non-measurable maps developed by Jorgen Hoffmann-Jorgensen ^{[Hoffmann-Jorgensen 1991]}, which underlies the $⇝$ convergence used throughout. David Pollard's 1984 monograph ^{[Pollard 1984]} made the chaining and symmetrisation arguments and the VC-class functional CLT standard, and the synthesis — bracketing and uniform-entropy CLTs, the preservation calculus, multiplier and bootstrap theorems, and the marriage with the functional delta method — was codified in van der Vaart and Wellner's 1996 monograph ^{[van der Vaart Wellner 1996]} and van der Vaart's 1998 text ^{[van der Vaart 1998]}, which fixed the formulation in $ℓ^{\infty} (F)$ now used in the field.

Bibliography Master

@article{Donsker1952,
  author  = {Donsker, Monroe D.},
  title   = {Justification and extension of {Doob's} heuristic approach to the {Kolmogorov-Smirnov} theorems},
  journal = {Annals of Mathematical Statistics},
  volume  = {23},
  number  = {2},
  year    = {1952},
  pages   = {277--281}
}

@article{Dudley1978,
  author  = {Dudley, Richard M.},
  title   = {Central limit theorems for empirical measures},
  journal = {Annals of Probability},
  volume  = {6},
  number  = {6},
  year    = {1978},
  pages   = {899--929}
}

@book{HoffmannJorgensen1991,
  author    = {Hoffmann-J{\o}rgensen, J{\o}rgen},
  title     = {Stochastic Processes on Polish Spaces},
  publisher = {Aarhus Universitet, Matematisk Institut},
  series    = {Various Publications Series 39},
  year      = {1991}
}

@book{Pollard1984,
  author    = {Pollard, David},
  title     = {Convergence of Stochastic Processes},
  publisher = {Springer-Verlag},
  series    = {Springer Series in Statistics},
  address   = {New York},
  year      = {1984}
}

@book{vanderVaartWellner1996donsker,
  author    = {van der Vaart, Aad W. and Wellner, Jon A.},
  title     = {Weak Convergence and Empirical Processes: With Applications to Statistics},
  publisher = {Springer-Verlag},
  series    = {Springer Series in Statistics},
  address   = {New York},
  year      = {1996}
}

@book{vanderVaart1998donsker,
  author    = {van der Vaart, Aad W.},
  title     = {Asymptotic Statistics},
  publisher = {Cambridge University Press},
  series    = {Cambridge Series in Statistical and Probabilistic Mathematics},
  year      = {1998}
}

@book{Dudley1999donsker,
  author    = {Dudley, Richard M.},
  title     = {Uniform Central Limit Theorems},
  publisher = {Cambridge University Press},
  series    = {Cambridge Studies in Advanced Mathematics 63},
  year      = {1999}
}

Prerequisites

45.05.02
37.03.03

Tier anchors

beginner: van der Vaart 1998 Asymptotic Statistics (Cambridge) §19.1 (the empirical process, informal); Wasserman 2004 All of Statistics (Springer) §7.2 (the empirical-process band picture); an everyday 'the whole estimation-error curve, rescaled, settling into one fixed random shape' picture, building on the Glivenko-Cantelli staircase of 45.05.02
intermediate: van der Vaart 1998 Asymptotic Statistics (Cambridge) §19.1-19.2 and Ch. 18 (the empirical process indexed by a class, asymptotic equicontinuity, Donsker classes); van der Vaart & Wellner 1996 Weak Convergence and Empirical Processes (Springer) §1.5, §2.1 (weak convergence in $ℓ^{\infty} (F)$ , the abstract empirical CLT)
master: van der Vaart 1998 Asymptotic Statistics (Cambridge) Ch. 18-19; van der Vaart & Wellner 1996 Weak Convergence and Empirical Processes (Springer) Part 2 (§2.1 the Donsker theorem, §2.3 maximal inequalities, §2.5 the bracketing and uniform-entropy central limit theorems); Dudley 1999 Uniform Central Limit Theorems (Cambridge) Ch. 3-6

References

van der Vaart — Asymptotic Statistics · Cambridge University Press, Cambridge Series in Statistical and Probabilistic Mathematics, 1998; Ch. 18 (stochastic convergence in metric spaces, weak convergence in $ℓ^{\infty}$ ), Ch. 19 (empirical processes): §19.1 Donsker classes and the empirical-process CLT, §19.2 entropy sufficient conditions
van der Vaart and Wellner — Weak Convergence and Empirical Processes · Springer Series in Statistics, 1996; §1.5 (weak convergence in $ℓ^{\infty} (F)$ , the Hoffmann-Jorgensen theory of outer expectations), §2.1 (the Donsker theorem and asymptotic equicontinuity), §2.3-2.5 (maximal inequalities, bracketing and uniform-entropy CLTs)
Donsker — Justification and extension of Doob's heuristic approach to the Kolmogorov-Smirnov theorems · Annals of Mathematical Statistics 23 (1952), 277-281 (the empirical-process Brownian-bridge limit)
Dudley — Central limit theorems for empirical measures · Annals of Probability 6 (1978), 899-929 (the abstract empirical CLT, uniform-entropy / Donsker classes)
Hoffmann-Jorgensen — Stochastic Processes on Polish Spaces · Aarhus Universitet, Various Publications Series 39, 1991 (outer expectations and weak convergence of non-measurable maps)
Pollard — Convergence of Stochastic Processes · Springer Series in Statistics, 1984 (Ch. 2-7: empirical processes, chaining, the functional CLT for VC classes)
Dudley — Uniform Central Limit Theorems · Cambridge University Press, Cambridge Studies in Advanced Mathematics 63, 1999; Ch. 3-6 (Donsker classes, metric entropy, the empirical CLT)

Estimated time

beginner: 20m
intermediate: 55m
master: 95m