The Ergodic Theorems: Birkhoff, von Neumann, and Kingman

Intuition Beginner

The strong law of large numbers handles a single, very clean situation: you average a long run of independent identical trials, and the average settles to the expected value. But the world is full of sequences that are not independent. The temperature on successive days, the position of a particle bouncing inside a box, the letters in a long stretch of English text — each entry depends on the ones before it. The ergodic theorems are the answer to a simple question: when does the running average of a dependent sequence still settle down to a fixed number?

The key idea is stationarity. A sequence is stationary when its statistical behaviour does not depend on where you start the clock: the chunk of the sequence you see between day 100 and day 200 looks, in distribution, exactly like the chunk between day 1000 and day 1100. Time has no preferred origin. Many real sequences with strong dependence are still stationary in this sense, and stationarity turns out to be the right replacement for independence.

Picture a stationary sequence as a single system evolving by one fixed rule, like a marble rolling on a frictionless table that reflects off the walls. Applying the rule shifts the whole system forward one step, and because the rule preserves the underlying "size" of regions, the long-run behaviour stays balanced. Birkhoff's theorem says: track any measurement of such a system and its time-average over a long run converges. Whether it converges to one universal number, or to a number depending on which run you got, is decided by whether the system is ergodic — whether it eventually visits everywhere, mixing the whole space rather than staying trapped in one region.

The takeaway: the ergodic theorems extend the law of large numbers from independent sequences to stationary ones, replacing the expected value by a time-average that converges for almost every run, and equals one universal number exactly when the system is ergodic.

Visual Beginner

Picture a single point hopping around the surface of a doughnut by a fixed rule, leaving a trail. If the rule is ergodic, the trail eventually covers the whole surface evenly; the fraction of time spent over any patch matches the area of that patch.

The winding trail is one long run of the system; the shaded patch is a region you might measure. The bars show that two different runs reach the same long-run average when the system is ergodic. If the system were not ergodic, the surface would split into separate zones and different runs could settle on different averages.

Worked example Beginner

We compute a time-average for the simplest interesting stationary system: rotating a circle by a fixed angle and tracking how often a point lands in the right half.

Step 1. The system. Take the circle as the numbers from $0$ up to $1$, wrapping around so that $1$ is the same point as $0$. The rule is: add $a=0.3$ and wrap around. Starting from $0.1$, the orbit is $0.1,0.4,0.7,1.0$ which wraps to $0.0$, then $0.3,0.6,0.9,0.2,0.5,0.8$, and so on.

Step 2. The measurement. We measure $1$ when the point is in the right half (between $0.5$ and $1$) and $0$ otherwise. The first ten positions $0.1,0.4,0.7,0.0,0.3,0.6,0.9,0.2,0.5,0.8$ give measurements $0,0,1,0,0,1,1,0,1,1$.

Step 3. The running average. Summing those ten measurements gives $5$, so the average over the first ten steps is $5/10=0.5$. Continue for many more steps and the average stays near $0.5$.

Step 4. What the theorem predicts. Because $0.3$ is not a simple fraction of the circle in a way that makes the orbit repeat, the orbit spreads out evenly over the whole circle. The right half is exactly half the circle, so the long-run fraction of time spent there is $0.5$. The time-average of our measurement converges to $0.5$, which is also the area of the region we were detecting.

What this tells us: for this ergodic rotation, the long-run time-average of "fraction of steps in the right half" equals the size of the right half, $0.5$. The average over time, taken along one orbit, reproduces the average over space. That equality of time-average and space-average is the whole content of the ergodic theorem in miniature.

Check your understanding Beginner

Formal definition Intermediate+

Throughout, $(\Omega ,\mathcal{F},\mathbb{P})$ is a probability space. The expectation $\mathbb{E}[f]={\int }_{\Omega }fd\mathbb{P}$ is the Lebesgue integral against $\mathbb{P}$, and $L^p(\mathbb{P})$ denotes the $L^p$ spaces of 02.07.06, with $L^2(\mathbb{P})$ the Hilbert space of square-integrable variables.

Definition (measure-preserving transformation). A map $T:\Omega \to \Omega$ is measurable if $T^{-1}A\in \mathcal{F}$ for all $A\in \mathcal{F}$, and measure-preserving if $\mathbb{P}(T^{-1}A)=\mathbb{P}(A)$ for all $A\in \mathcal{F}$. The quadruple $(\Omega ,\mathcal{F},\mathbb{P},T)$ is then a measure-preserving system. Its associated Koopman operator $U_T$ acts on functions by $(U_{T}f)(\omega )=f(T\omega )$; measure-preservation is equivalent to $U_T$ being an isometry of $L^p(\mathbb{P})$ for every $p$, since $\mathbb{E}[|f\circ T{|}^p]=\mathbb{E}[|f{|}^p]$.

Definition (stationary sequence). A sequence $({\xi }_n{)}_{n\ge 0}$ of random variables is stationary if for every $k\ge 0$ the shifted block $({\xi }_k,{\xi }_{k+1},\dots )$ has the same joint distribution as $({\xi }_0,{\xi }_1,\dots )$. Every stationary sequence arises from a measure-preserving system: set $\Omega ={\mathbb{R}}^{\{0,1,\dots \}}$ with the law of $({\xi }_n)$, let $T$ be the shift $(T\omega {)}_n={\omega }_{n+1}$, and let $f(\omega )={\omega }_0$ be the zeroth coordinate, so that ${\xi }_n=f\circ T^n$. An i.i.d. sequence is the special case where $\mathbb{P}$ is a product measure.

Definition (invariant $\sigma$-algebra and ergodicity). A set $A\in \mathcal{F}$ is invariant if $T^{-1}A=A$, and almost invariant if $\mathbb{P}(T^{-1}A△A)=0$. The almost-invariant sets form a sub-$\sigma$-algebra $\mathcal{I}$, the invariant $\sigma$-algebra. The system is ergodic if every almost-invariant set has $\mathbb{P}(A)\in \{0,1\}$, equivalently if every $\mathcal{I}$-measurable function is a.s. constant. A function $g$ is invariant if $g\circ T=g$ a.s.; ergodicity says all invariant functions are a.s. constant.

Definition (subadditive sequence). A doubly indexed family $(X_{m,n}{)}_{0\le m<n}$ of integrable variables on a measure-preserving system is subadditive if $$X_{0,n}\le X_{0,m}+X_{m,n}(0<m<n),X_{m,n}=X_{0,n-m}\circ T^m,$$ the second relation being stationarity of the increments. The time constant is $\gamma ={\inf }_{n\ge 1}\mathbb{E}[X_{0,n}]/n$, finite provided $\mathbb{E}[X_{0,n}]\ge -cn$ for some constant $c$. Choosing $X_{0,n}={\sum }_{k=0}^{n-1}f\circ T^k$ makes the subadditivity an equality and recovers the additive (Birkhoff) setting.

Counterexamples to common slips Intermediate+

• Stationary is not i.i.d. The sequence ${\xi }_n=Z$ for a single random variable $Z$ (every term equal) is stationary but maximally dependent. Birkhoff still applies; the time-average is $Z$ itself, an invariant non-constant limit. The system is not ergodic, which is exactly why the limit is a random variable rather than a constant.

• Measure-preserving is about preimages, not images. The condition is $\mathbb{P}(T^{-1}A)=\mathbb{P}(A)$. For non-invertible $T$ the forward image $\mathbb{P}(TA)$ can differ from $\mathbb{P}(A)$; the doubling map $\omega \mapsto 2\omega \mathrm{mod}1$ on $[0,1)$ preserves Lebesgue measure through preimages while being two-to-one.

• Ergodicity is a property of the pair $(T,\mathbb{P})$, not of $T$ alone. The same map can be ergodic for one invariant measure and not another. An irrational rotation is ergodic for Lebesgue measure but not for a point mass on a periodic orbit (there are none) — and a rational rotation is ergodic for no atomless invariant measure at all, since its orbits are finite.

• The invariant $\sigma$-algebra need not be degenerate. For the shift on a product space the invariant $\sigma$-algebra is the tail/exchangeable field, which the Kolmogorov zero-one law 37.02.01 forces to be $\mathbb{P}$-degenerate (every set has probability $0$ or $1$) — that is why i.i.d. sequences are ergodic. Drop independence and $\mathcal{I}$ can be rich; the conditional expectation $\mathbb{E}[f\mid \mathcal{I}]$ is then a genuine random limit.

• Subadditivity is one-directional. From $X_{0,n}\le X_{0,m}+X_{m,n}$ one gets $\mathbb{E}[X_{0,n}]/n\to \gamma ={\inf }_n\mathbb{E}[X_{0,n}]/n$ by Fekete's lemma, but the pointwise limit equals $\gamma$ only after the ergodic-theoretic argument; assuming the pointwise limit is the naive $\lim \mathbb{E}[X_{0,n}]/n$ without proof is the slip Kingman's theorem repairs.

Key theorem with proof Intermediate+

Theorem (Birkhoff's pointwise ergodic theorem; Birkhoff 1931). Let $(\Omega ,\mathcal{F},\mathbb{P},T)$ be a measure-preserving system and $f\in L^1(\mathbb{P})$. Then the Birkhoff averages $$A_{n}f=\frac{1}{n}\sum _{k=0}^{n-1}f\circ T^k$$ converge almost surely and in $L^1$ to $\mathbb{E}[f\mid \mathcal{I}]$, the conditional expectation of $f$ on the invariant $\sigma$-algebra. If $T$ is ergodic the limit is the constant $\mathbb{E}[f]$.

The analytic core is the maximal ergodic lemma; the pointwise statement follows by a squeeze on the limsup and liminf of the averages.

Lemma (maximal ergodic lemma). For $f\in L^1$ write $S_{0}f=0$ and $S_{k}f={\sum }_{j=0}^{k-1}f\circ T^j$ for $k\ge 1$, and $M_{n}f={\max }_{0\le k\le n}{S}_{k}f$. Let $E_n=\{M_{n}f>0\}$. Then $${\int }_{E_n}fd\mathbb{P}\ge 0.$$

Proof. On $E_n$, for $0\le k\le n$ we have $M_{n}f\ge S_{k}f$, hence $M_{n}f\ge S_{k+1}f-f\circ T^0$ rearranges through $S_{k+1}f=f+(S_{k}f)\circ T$. Concretely, for $1\le k\le n$, $S_{k}f=f+(S_{k-1}f)\circ T\le f+(M_{n}f)\circ T$, and also $S_{0}f=0$. Taking the maximum over $0\le k\le n$ gives $M_{n}f\le f+(M_{n}f{)}^{+}\circ T$ pointwise, because $(M_{n}f)\circ T\le (M_{n}f{)}^{+}\circ T$ and the $k=0$ term is dominated by $f+(M_{n}f{)}^{+}\circ T$ when $f\ge -((M_{n}f{)}^{+}\circ T)$, which holds since the right side is the bound forced on every $S_{k}f$. Therefore on $E_n$, where $M_{n}f>0$ so $M_{n}f=(M_{n}f{)}^{+}$, $$f\ge (M_{n}f{)}^{+}-(M_{n}f{)}^{+}\circ T.$$ Integrate over $E_n$ and extend the right side to all of $\Omega$ using $(M_{n}f{)}^{+}=0$ off $E_n$: $${\int }_{E_n}fd\mathbb{P}\ge {\int }_{\Omega }(M_{n}f{)}^{+}d\mathbb{P}-{\int }_{E_n}(M_{n}f{)}^{+}\circ Td\mathbb{P}\ge {\int }_{\Omega }(M_{n}f{)}^{+}d\mathbb{P}-{\int }_{\Omega }(M_{n}f{)}^{+}\circ Td\mathbb{P}.$$ By measure-preservation $\int (M_{n}f{)}^{+}\circ Td\mathbb{P}=\int (M_{n}f{)}^{+}d\mathbb{P}$, so the right side is $0$. $\square$

Proof of the Theorem. Replacing $f$ by $f-\mathbb{E}[f\mid \mathcal{I}]$ reduces to the case $\mathbb{E}[f\mid \mathcal{I}]=0$, for which we must show $A_{n}f\to 0$ a.s. Set $\bar{f}={\mathrm{limsup}}_n{A}_{n}f$; since $A_{n}f\circ T$ and $A_{n}f$ differ by $(f\circ T^n-f)/n\to 0$ in the relevant sense, $\bar{f}$ is invariant, hence $\mathcal{I}$-measurable. Fix $\epsilon >0$ and let $B=\{\bar{f}>\epsilon \}\in \mathcal{I}$. Apply the maximal ergodic lemma to $g=(f-\epsilon )1_B$, which is invariant-set-restricted so that its Birkhoff sums on $B$ coincide with those of $f-\epsilon$. The lemma gives ${\int }_{B\cap \{{\sup }_k{S}_{k}g>0\}}gd\mathbb{P}\ge 0$; on $B$, ${\sup }_k{S}_k(f-\epsilon )>0$ holds because $\bar{f}>\epsilon$ there forces some average $A_{k}f>\epsilon$. Hence $$0\le {\int }_B(f-\epsilon )d\mathbb{P}={\int }_{B}fd\mathbb{P}-\epsilon \mathbb{P}(B)={\int }_B\mathbb{E}[f\mid \mathcal{I}]d\mathbb{P}-\epsilon \mathbb{P}(B)=-\epsilon \mathbb{P}(B),$$ using $B\in \mathcal{I}$ and $\mathbb{E}[f\mid \mathcal{I}]=0$. Thus $\epsilon \mathbb{P}(B)\le 0$, forcing $\mathbb{P}(B)=0$; as $\epsilon$ was arbitrary, $\bar{f}\le 0$ a.s. Applying the same argument to $-f$ gives ${\mathrm{liminf}}_n{A}_{n}f\ge 0$ a.s., so $A_{n}f\to 0$ a.s. The $L^1$ convergence follows from a.s. convergence plus uniform integrability of $(A_{n}f)$, which holds because the family $\{f\circ T^k\}$ is uniformly integrable (each has the same distribution as $f\in L^1$) and Cesàro averages of a uniformly integrable family are uniformly integrable 37.04.03. $\square$

Bridge. Birkhoff's theorem builds toward the entire theory of stationary sequences and appears again in the subadditive theorem below, where additive Birkhoff sums are the boundary case of a subadditive family. The foundational reason the average converges is that the invariant $\sigma$-algebra carries every limit: the time-average can only land on an $\mathcal{I}$-measurable function, and the maximal ergodic lemma pins that function to be $\mathbb{E}[f\mid \mathcal{I}]$. This is exactly the conditional-expectation structure of 37.04.03, now read dynamically: the projection onto invariants is the conditional expectation, which is why the mean (von Neumann) theorem below is the $L^2$ shadow of the same fact and is dual to the pointwise statement. Putting these together, the i.i.d. strong law 37.02.02 is the case where $T$ is the product shift and the zero-one law 37.02.01 collapses $\mathcal{I}$ to the constants, so $\mathbb{E}[f\mid \mathcal{I}]=\mathbb{E}[f]$; the central insight is that independence is one route to a degenerate invariant field, and stationarity-plus-ergodicity is the general route. The bridge is the identification of "time-average" with "projection onto the invariant $\sigma$-algebra".

Exercises Intermediate+

Advanced results Master

Theorem 1 (von Neumann mean ergodic theorem; von Neumann 1932). Let $U$ be a contraction of a Hilbert space $H$ — in particular the Koopman isometry $U_T$ of $L^2(\mathbb{P})$. Then $\frac{1}{n}{\sum }_{k=0}^{n-1}{U}^k\to P$ strongly, where $P$ is the orthogonal projection onto the invariant subspace $\ker (I-U)$. For $U=U_T$ the projection is conditional expectation onto $\mathcal{I}$, so $A_{n}f\to \mathbb{E}[f\mid \mathcal{I}]$ in $L^2$. The mean theorem is the Hilbert-space shadow of Birkhoff: it gives $L^2$ rather than pointwise convergence but extends to any contraction and requires only the orthogonal decomposition $H=\ker (I-U)\oplus \overline{\text{range}(I-U)}$ ^{[von Neumann 1932]}.

Theorem 2 (Birkhoff pointwise ergodic theorem; Birkhoff 1931). For $f\in L^1$ and any measure-preserving $T$, $A_{n}f\to \mathbb{E}[f\mid \mathcal{I}]$ almost surely and in $L^1$. The maximal ergodic lemma is the irreducible analytic content; the pointwise statement strictly strengthens the mean theorem on $L^1\cap L^2$ but neither contains the other in general, since $L^1$ convergence and a.s. convergence are distinct modes ^{[Birkhoff 1931]}.

Theorem 3 (ergodic decomposition). Every invariant probability measure for a measurable transformation on a standard Borel space is a mixture (a barycentre) of ergodic invariant measures: there is a probability measure on the set of ergodic measures such that $\mathbb{P}=\int {\mathbb{P}}_{e}d\rho (e)$. Consequently the Birkhoff limit $\mathbb{E}[f\mid \mathcal{I}]$ evaluated at $\omega$ is the integral of $f$ against the ergodic component through $\omega$. This is the structural reason the non-ergodic case reduces to the ergodic one: $\mathcal{I}$ indexes the ergodic components.

Theorem 4 (Kingman's subadditive ergodic theorem; Kingman 1968). Let $(X_{m,n})$ be a subadditive family on a measure-preserving system with $\mathbb{E}[X_{0,1}^{+}]<\infty$ and ${\inf }_n\mathbb{E}[X_{0,n}]/n=:\gamma >-\infty$. Then $X_{0,n}/n\to X$ a.s. and in $L^1$, where $X$ is an invariant random variable with $\mathbb{E}[X]=\gamma$; if $T$ is ergodic, $X=\gamma$ a.s. The additive case $X_{0,n}={\sum }_{k<n}f\circ T^k$ is Birkhoff. The standard modern proof (Steele's variant of Kingman, via the Burkholder-style decomposition or the Ackoglu-Krengel superadditive approach) controls $\mathrm{limsup}{X}_{0,n}/n$ by comparison against truncated additive cocycles and identifies the limit through the inf-formula ^{[Kingman 1968]}.

Theorem 5 (Furstenberg-Kesten / multiplicative ergodic; Furstenberg-Kesten 1960). Let $(M_k)$ be a stationary ergodic sequence of $d\times d$ random matrices with $\mathbb{E}{\log }^{+}\Vert M_1\Vert <\infty$. Then $\frac{1}{n}\log \Vert M_n\cdots M_1\Vert \to {\gamma }_1$ a.s., the top Lyapunov exponent, equal to ${\inf }_n\frac{1}{n}\mathbb{E}\log \Vert M_n\cdots M_1\Vert$. This is Kingman applied to $X_{0,n}=\log \Vert M_n\cdots M_1\Vert$, submultiplicativity supplying subadditivity. Oseledets' multiplicative ergodic theorem refines this to a full Lyapunov spectrum ${\gamma }_1\ge \cdots \ge {\gamma }_d$ with an associated measurable filtration ^{[Furstenberg-Kesten 1960]}.

Theorem 6 (longest increasing subsequence and first-passage percolation). For a uniform random permutation of $\{1,\dots ,n\}$, the length $L_n$ of the longest increasing subsequence satisfies $L_n/\sqrt{n}\to 2$ a.s.; the subadditive structure (after Hammersley's Poissonised model) yields existence of the constant via Kingman, and the value $2$ comes from the later Vershik-Kerov / Logan-Shepp analysis. In first-passage percolation on ${\mathbb{Z}}^d$ with i.i.d. non-negative edge weights, the passage time $T(0,nx)$ satisfies $T(0,nx)/n\to \mu (x)$ a.s. for a deterministic norm-like time constant $\mu$, again by subadditivity. These are the canonical combinatorial applications of Kingman's theorem ^{[Hammersley-Welsh 1965]}.

Synthesis. The four ergodic theorems are one circle of ideas seen at different resolutions, and the foundational reason they cohere is that each identifies a limit of averages with a projection onto the invariant $\sigma$-algebra. The mean theorem is the $L^2$ projection $P=\mathbb{E}[\cdot \mid \mathcal{I}]$ obtained from the orthogonal splitting $H=\ker (I-U)\oplus \overline{\text{range}(I-U)}$; the pointwise theorem is exactly the a.s. upgrade of that same projection, the maximal ergodic lemma supplying the control that $L^2$ geometry cannot. This is precisely the conditional-expectation and uniform-integrability machinery of 37.04.03 read dynamically, and it is dual to the martingale convergence theorem, where the limiting $\sigma$-algebra is a filtration tail rather than an invariant field. Putting these together, the i.i.d. strong law 37.02.02 is the ergodic special case in which the Kolmogorov zero-one law 37.02.01 makes $\mathcal{I}$ $\mathbb{P}$-degenerate, so the central insight — independence is one mechanism producing a degenerate invariant field — generalises to stationarity-plus-ergodicity. The subadditive theorem generalises the additive Birkhoff statement to families that only sub-add, and this single generalisation is what unlocks Lyapunov exponents, longest increasing subsequences, and first-passage percolation: the bridge is that submultiplicativity of norms becomes subadditivity of logs, and Kingman converts it into an a.s. growth rate equal to the inf-formula $\gamma ={\inf }_n\mathbb{E}[X_{0,n}]/n$.

Full proof set Master

Proposition 1 (Fekete's lemma underlies the time constant). If $(a_n)$ is a real sequence with $a_{m+n}\le a_m+a_n$ for all $m,n\ge 1$, then ${\lim }_n{a}_n/n={\inf }_n{a}_n/n\in [-\infty ,\infty )$.

Proof. Let $\gamma ={\inf }_n{a}_n/n$. Fix $\epsilon >0$ and choose $m$ with $a_m/m<\gamma +\epsilon$. For $n>m$ write $n=qm+r$ with $0\le r<m$. Subadditivity gives $a_n\le q{a}_m+a_r$ (with $a_0:=0$), so $$\frac{a_n}{n}\le \frac{q{a}_m}{qm+r}+\frac{a_r}{n}\le \frac{a_m}{m}+\frac{{\max }_{0\le r<m}{a}_r}{n}.$$ As $n\to \infty$ the last term vanishes, so ${\mathrm{limsup}}_n{a}_n/n\le a_m/m<\gamma +\epsilon$. Since $\epsilon$ is arbitrary, ${\mathrm{limsup}}_n{a}_n/n\le \gamma \le {\mathrm{liminf}}_n{a}_n/n$, giving equality. $\square$

Proposition 2 (the maximal ergodic lemma yields the Hardy-Littlewood maximal bound). With $f\in L^1$ and $f^{\ast }={\sup }_{n\ge 1}{A}_{n}f$, for every $\lambda >0$, $$\mathbb{P}(f^{\ast }>\lambda )\le \frac{1}{\lambda }\mathbb{E}[|f|1_{\{f^{\ast }>\lambda \}}]\le \frac{\mathbb{E}|f|}{\lambda }.$$

Proof. Apply the maximal ergodic lemma to $g=f-\lambda$ on the invariant-restricted set $\{f^{\ast }>\lambda \}$. The event $\{f^{\ast }>\lambda \}$ means some average $A_{n}f>\lambda$, equivalently some partial sum $S_n(f-\lambda )>0$, so $\{f^{\ast }>\lambda \}\subseteq {\bigcup }_n\{M_{n}g>0\}$. The lemma gives ${\int }_{\{M_{n}g>0\}}(f-\lambda )d\mathbb{P}\ge 0$ for each $n$; letting $n\to \infty$ and using monotone convergence on the increasing sets, ${\int }_{\{f^{\ast }>\lambda \}}(f-\lambda )d\mathbb{P}\ge 0$, i.e. $\lambda \mathbb{P}(f^{\ast }>\lambda )\le {\int }_{\{f^{\ast }>\lambda \}}fd\mathbb{P}\le \mathbb{E}[|f|1_{\{f^{\ast }>\lambda \}}]$. Dividing by $\lambda$ and bounding the indicator by $1$ gives the weak-type bound. $\square$

Proposition 3 (ergodicity is equivalent to a mixing-free averaging criterion). A measure-preserving system is ergodic if and only if for all $A,B\in \mathcal{F}$, $$\frac{1}{n}\sum _{k=0}^{n-1}\mathbb{P}(T^{-k}A\cap B)\to \mathbb{P}(A)\mathbb{P}(B).$$

Proof. Suppose $T$ is ergodic. Apply Birkhoff to $f=1_A$: $A_n{1}_A\to \mathbb{P}(A)$ a.s. and in $L^1$. Multiply by $1_B$ and take expectations, using bounded convergence: $\frac{1}{n}{\sum }_{k<n}\mathbb{E}[1_A\circ T^k{1}_B]\to \mathbb{P}(A)\mathbb{E}[1_B]=\mathbb{P}(A)\mathbb{P}(B)$, and $\mathbb{E}[1_A\circ T^k{1}_B]=\mathbb{P}(T^{-k}A\cap B)$. Conversely, suppose the averaging criterion holds and let $A$ be invariant, $T^{-1}A=A$. Take $B=A$: then $T^{-k}A\cap A=A$ for all $k$, so the left side is constantly $\mathbb{P}(A)$, while the right side is $\mathbb{P}(A{)}^2$; hence $\mathbb{P}(A)=\mathbb{P}(A{)}^2$, forcing $\mathbb{P}(A)\in \{0,1\}$. So every invariant set is $\mathbb{P}$-degenerate and $T$ is ergodic. $\square$

Proposition 4 (subadditive limit has the stated mean). Under the hypotheses of Kingman's theorem, the a.s. limit $X={\lim }_n{X}_{0,n}/n$ satisfies $\mathbb{E}[X]=\gamma ={\inf }_n\mathbb{E}[X_{0,n}]/n$.

Proof. By Proposition 1 applied to $a_n=\mathbb{E}[X_{0,n}]$ (subadditive in $n$ by the cocycle relation and $\mathbb{E}[X_{m,n}]=\mathbb{E}[X_{0,n-m}]$), $\mathbb{E}[X_{0,n}]/n\to \gamma$. Kingman's theorem asserts $X_{0,n}/n\to X$ in $L^1$ as well as a.s.; $L^1$ convergence implies convergence of means, so $\mathbb{E}[X]={\lim }_n\mathbb{E}[X_{0,n}/n]=\gamma$. Invariance of $X$ follows from $X={\lim }_n{X}_{0,n}/n$ and the cocycle relation $X_{0,n}=X_{0,1}+X_{1,n}$ with $X_{1,n}=X_{0,n-1}\circ T$, giving $X=X\circ T$ a.s. after dividing by $n$ and letting $n\to \infty$. $\square$

Connections Master

• The strong law of large numbers 37.02.02 is the ergodic special case of Birkhoff's theorem: realise the i.i.d. sequence as coordinates on a product space, take $T$ the shift and $f$ the zeroth coordinate, and the Kolmogorov zero-one law makes the invariant $\sigma$-algebra $\mathbb{P}$-degenerate so that $\mathbb{E}[f\mid \mathcal{I}]$ collapses to $\mathbb{E}[X_1]$. This unit is the stationary-sequence generalisation that this dependency makes precise.

• The Borel-Cantelli lemmas and Kolmogorov zero-one law 37.02.01 supply the reason i.i.d. shifts are ergodic: the invariant $\sigma$-algebra sits inside the tail field, which the zero-one law makes $\mathbb{P}$-degenerate. Ergodicity is thus the dynamical face of the zero-one law, and the non-ergodic case is what happens when the tail field is replaced by a rich invariant field.

• Doob's inequalities, uniform integrability, and $L^p$ convergence 37.04.03 furnish the $L^1$ half of every ergodic theorem here: the Birkhoff and Kingman limits hold in $L^1$ because the Cesàro averages of an identically-distributed family are uniformly integrable, and the maximal ergodic lemma is the additive-cocycle analogue of Doob's maximal inequality for martingales.

• The $L^p$ and Hilbert-space theory 02.07.06 is the arena for von Neumann's mean theorem: the Koopman operator is an $L^2$ isometry, the invariant functions form a closed subspace, and the orthogonal decomposition $L^2=\ker (I-U_T)\oplus \overline{\text{range}(I-U_T)}$ is the Riesz-Fischer completeness of $L^2$ at work.

• The ergodic theorem for Markov chains 37.05.07 is the Markovian instance: a positive-recurrent irreducible chain is a stationary ergodic sequence under its invariant law, and its time-average theorem $\frac{1}{n}{\sum }_{k<n}f(X_k)\to {\mathbb{E}}_{\pi }[f]$ is Birkhoff specialised to the shift on path space, with the excursion decomposition replacing the abstract maximal lemma.

The dynamical-systems treatment of ergodic theory — mixing, weak mixing, entropy (Kolmogorov-Sinai), the isomorphism theory of Ornstein, and smooth ergodic theory (Pesin theory, SRB measures) — is a separate forthcoming curriculum spine. This unit deliberately keeps its emphasis on the probabilistic limit theorems for stationary sequences: Birkhoff, von Neumann, and Kingman as the stationary-sequence generalisation of the strong law of large numbers, rather than on the structural dynamics that those theorems also serve.

Historical & philosophical context Master

The ergodic theorems arose from a foundational problem in statistical mechanics: Boltzmann's ergodic hypothesis of the 1870s posited that a mechanical system's long-time average along a single trajectory equals its average over the constant-energy surface, justifying the replacement of intractable time-averages by computable phase-space averages. The hypothesis as literally stated — that a single orbit visits every point of the energy surface — is false on dimensional grounds, and the corrected quasi-ergodic hypothesis (the orbit is dense) was still too weak to license the averaging. The resolution came in 1931-1932 in two papers in the Proceedings of the National Academy of Sciences. John von Neumann's Proof of the quasi-ergodic hypothesis ^{[von Neumann 1932]} established the mean ($L^2$) ergodic theorem first, using the spectral theory of unitary operators he had just developed; George Birkhoff's Proof of the ergodic theorem ^{[Birkhoff 1931]} then proved the stronger pointwise statement. Birkhoff's paper appeared in print before von Neumann's despite von Neumann having the result earlier, a priority episode documented in the correspondence reproduced by later historians of the period.

The decisive conceptual move was the recognition, sharpened over the following two decades by Khinchin, Hopf, and Kolmogorov, that the right hypothesis is not a property of individual orbits but the measure-theoretic degeneracy of the invariant $\sigma$-algebra. This reframed ergodicity as an indecomposability condition on an invariant measure and tied it directly to the law of large numbers: i.i.d. sequences are exactly the ergodic stationary sequences whose invariant field is $\mathbb{P}$-degenerate by the zero-one law.

John Kingman's 1968 Journal of the Royal Statistical Society paper ^{[Kingman 1968]} introduced the subadditive ergodic theorem, motivated by problems where the natural quantities only sub-add rather than add — products of random matrices studied by Furstenberg and Kesten in 1960 ^{[Furstenberg-Kesten 1960]}, and the first-passage percolation and subadditive-process framework of Hammersley and Welsh in 1965 ^{[Hammersley-Welsh 1965]}. Kingman's theorem made the existence of Lyapunov exponents, time constants in percolation, and the longest-increasing-subsequence growth rate immediate consequences of a single subadditivity principle, and Steele and others subsequently gave shorter proofs that became the textbook standard.

Bibliography Master

@article{Birkhoff1931,
  author  = {Birkhoff, George D.},
  title   = {Proof of the ergodic theorem},
  journal = {Proceedings of the National Academy of Sciences},
  volume  = {17},
  number  = {12},
  year    = {1931},
  pages   = {656--660}
}

@article{vonNeumann1932,
  author  = {von Neumann, John},
  title   = {Proof of the quasi-ergodic hypothesis},
  journal = {Proceedings of the National Academy of Sciences},
  volume  = {18},
  number  = {1},
  year    = {1932},
  pages   = {70--82}
}

@article{Kingman1968,
  author  = {Kingman, John F. C.},
  title   = {The ergodic theory of subadditive stochastic processes},
  journal = {Journal of the Royal Statistical Society, Series B},
  volume  = {30},
  number  = {3},
  year    = {1968},
  pages   = {499--510}
}

@article{FurstenbergKesten1960,
  author  = {Furstenberg, Harry and Kesten, Harry},
  title   = {Products of random matrices},
  journal = {Annals of Mathematical Statistics},
  volume  = {31},
  number  = {2},
  year    = {1960},
  pages   = {457--469}
}

@incollection{HammersleyWelsh1965,
  author    = {Hammersley, John M. and Welsh, Dominic J. A.},
  title     = {First-passage percolation, subadditive processes, stochastic networks, and generalized renewal theory},
  booktitle = {Bernoulli-Bayes-Laplace Anniversary Volume},
  publisher = {Springer},
  year      = {1965},
  pages     = {61--110}
}

@book{Walters1982,
  author    = {Walters, Peter},
  title     = {An Introduction to Ergodic Theory},
  publisher = {Springer},
  series    = {Graduate Texts in Mathematics},
  volume    = {79},
  year      = {1982}
}

@book{Krengel1985,
  author    = {Krengel, Ulrich},
  title     = {Ergodic Theorems},
  publisher = {de Gruyter},
  year      = {1985}
}

@book{Steele1997,
  author    = {Steele, J. Michael},
  title     = {Probability Theory and Combinatorial Optimization},
  publisher = {SIAM},
  series    = {CBMS-NSF Regional Conference Series},
  year      = {1997}
}

@book{Durrett2019,
  author    = {Durrett, Rick},
  title     = {Probability: Theory and Examples},
  edition   = {5},
  publisher = {Cambridge University Press},
  year      = {2019}
}

@book{Kallenberg2002,
  author    = {Kallenberg, Olav},
  title     = {Foundations of Modern Probability},
  edition   = {2},
  publisher = {Springer},
  year      = {2002}
}