Probability Spaces and the Kolmogorov Extension Theorem

Intuition Beginner

Probability begins with a bookkeeping question: before any experiment runs, what could possibly happen, and how should we weigh each possibility? A probability space is the answer written down carefully. It has three parts. The first is a list of every outcome the experiment could produce, called the sample space. The second is a collection of questions we are allowed to ask about the outcome, called events, where each event is a bundle of outcomes. The third is a rule that assigns each event a weight between zero and one, the probability, with the whole sample space weighing exactly one.

Think of rolling a die. The sample space is the six faces. An event might be "the result is even," which bundles together the faces two, four, and six. The weight rule gives this event the value one half for a fair die. The two requirements that make this honest bookkeeping are that the weight of an impossible event is zero, the weight of the certain event is one, and the weight of a bundle made from separate non-overlapping events is the sum of their individual weights.

The subtle part is what happens with infinitely many outcomes. If you flip a coin forever, a single outcome is an entire endless sequence of heads and tails, and there are uncountably many such sequences. You cannot weigh each one individually and add them up. Instead you only ever directly weigh events that depend on finitely many flips at a time, such as "the first three flips are heads." These finite-window events are called cylinder events, because they pin down a few coordinates and leave the rest free.

The Kolmogorov extension theorem is the promise that this works. If you specify the weights for every finite window in a way that does not contradict itself, then there is one and only one weight rule on the full infinite space of sequences that agrees with all of your finite-window weights at once. The non-contradiction requirement is called consistency: the weights you assign to the first three flips must match what you get from the weights on the first four flips after ignoring the fourth.

The one-sentence takeaway: a probability space is a sample space plus a list of askable events plus a weight rule summing to one, and the Kolmogorov extension theorem builds the weight rule on an infinite space out of consistent weights on finite windows.

Visual Beginner

Picture an infinite sequence of coin flips as an unending row of slots, each slot eventually filled with H or T. You can never see the whole row at once, but you can put a window over the first few slots.

Top: the infinite sequence with a finite window over the first three coordinates; the pattern inside the window is a cylinder event. Bottom: the eight weights for three-slot patterns sum to one. The arrow shows consistency: widening the window to four slots and then summing over the last slot must return the original three-slot weights.

The picture behind the extension theorem is that all these finite-window weight charts, one for each window size, fit together like nested rulers. Consistency is the statement that the nested rulers agree wherever they overlap, and the theorem says the nested rulers determine a single weight rule on the whole infinite row.

Worked example Beginner

We build the probability rule for two independent fair coin flips and check it extends a third flip consistently.

Step 1. The sample space for two flips is the four patterns HH, HT, TH, TT. Each flip is fair and the two flips are independent, so each pattern has weight one quarter: the weight of a two-flip pattern is the weight of its first flip, one half, times the weight of its second flip, one half, giving one quarter.

Step 2. Check the event "first flip is heads." This event bundles HH and HT. Its weight is one quarter plus one quarter, which is one half. That matches the weight of a single heads on the first flip, as it must.

Step 3. Now widen to three flips. The eight patterns each get weight one eighth, again because each is one half times one half times one half.

Step 4. Verify consistency between the three-flip weights and the two-flip weights. Take the two-flip pattern HT. In the three-flip world it corresponds to the two patterns HTH and HTT, since the third flip is free. Their weights are one eighth plus one eighth, which is one quarter. That equals the original two-flip weight of HT.

Step 5. Read off the conclusion. The three-flip weights, when you ignore the third flip by summing over its two values, reproduce the two-flip weights exactly. The family of weights for every window size agrees on overlaps, so it is consistent.

What this tells us: the finite-window weights for independent fair coins are consistent, so the extension theorem guarantees one weight rule on the infinite space of all coin-flip sequences. That rule is the probability model for flipping a coin forever, and it is built entirely from the simple finite-window weights we just checked.

Check your understanding Beginner

Formal definition Intermediate+

Definition (probability space). A probability space is a triple $(\Omega ,\mathcal{F},P)$ where $\Omega$ is a set (the sample space), $\mathcal{F}$ is a $\sigma$-algebra of subsets of $\Omega$ 02.07.01 (the events), and $P:\mathcal{F}\to [0,1]$ is a probability measure: a countably additive set-function with $P(\Omega )=1$. Countable additivity means $P({⨆}_n{A}_n)={\sum }_{n}P(A_n)$ for any countable disjoint family $A_n\in \mathcal{F}$. A probability space is a normalized measure space, so the entire integration apparatus of measure theory applies with total mass one.

Definition (random variable and its law). Given a measurable space $(E,\mathcal{E})$, a random variable with values in $E$ is a measurable map $X:(\Omega ,\mathcal{F})\to (E,\mathcal{E})$, that is, $X^{-1}(B)\in \mathcal{F}$ for every $B\in \mathcal{E}$. The law (or distribution) of $X$ is the pushforward measure $P_X=X_{\ast }P=P\circ X^{-1}$ on $(E,\mathcal{E})$, defined by $P_X(B)=P(X^{-1}(B))=P(X\in B)$. The pushforward is a probability measure on $E$, and the change-of-variables identity ${\int }_{E}fd{P}_X={\int }_{\Omega }(f\circ X)dP$ holds for every non-negative measurable $f:E\to [0,\infty ]$.

Definition (independence of $\sigma$-algebras). Sub-$\sigma$-algebras ${\mathcal{G}}_1,\dots ,{\mathcal{G}}_n\subseteq \mathcal{F}$ are independent if $P(A_1\cap \cdots \cap A_n)={\prod }_{i=1}^{n}P(A_i)$ for every choice $A_i\in {\mathcal{G}}_i$. An arbitrary family $({\mathcal{G}}_i{)}_{i\in I}$ is independent if every finite subfamily is. Random variables $(X_i{)}_{i\in I}$ are independent when the generated $\sigma$-algebras $\sigma (X_i)$ are independent; equivalently the joint law of any finite subfamily is the product 02.07.07 of the marginal laws.

Definition (cylinder $\sigma$-algebra on a product). Let $(E_t,{\mathcal{E}}_t{)}_{t\in T}$ be a family of measurable spaces indexed by a set $T$, and let $\Omega ={\prod }_{t\in T}{E}_t$ with coordinate projections ${\pi }_t:\Omega \to E_t$. For a finite subset $S=\{t_1,\dots ,t_n\}\subseteq T$ write ${\pi }_S=({\pi }_{t_1},\dots ,{\pi }_{t_n}):\Omega \to {\prod }_{t\in S}{E}_t=E_S$. A cylinder set with base $S$ is a set ${\pi }_S^{-1}(B)$ for $B\in {\mathcal{E}}_S={⨂}_{t\in S}{\mathcal{E}}_t$. The cylinder sets over all finite $S$ form an algebra $\mathcal{A}$. The cylinder (or product) $\sigma$-algebra is ${\mathcal{E}}_T={⨂}_{t\in T}{\mathcal{E}}_t=\sigma (\mathcal{A})$, the smallest $\sigma$-algebra making every ${\pi }_t$ measurable.

Definition (finite-dimensional distributions and consistency). A finite-dimensional distribution (f.d.d.) family is a collection of probability measures ${\mu }_S$ on $(E_S,{\mathcal{E}}_S)$, one for each finite $S\subseteq T$. The family is consistent (or projective) if it is stable under marginalization: whenever $S\subseteq S^{\prime }$ are finite, the canonical projection $p_{S^{\prime }S}:E_{S^{\prime }}\to E_S$ satisfies $(p_{S^{\prime }S}{)}_{\ast }{\mu }_{S^{\prime }}={\mu }_S$. Concretely, integrating out the extra coordinates of ${\mu }_{S^{\prime }}$ returns ${\mu }_S$.

Counterexamples to common slips Intermediate+

• The cylinder $\sigma$-algebra is much smaller than the power set. For $T$ uncountable, every set in ${\mathcal{E}}_T$ depends on only countably many coordinates: if $A\in {\mathcal{E}}_T$ there is a countable $T_0\subseteq T$ with $A={\pi }_{T_0}^{-1}({\pi }_{T_0}(A))$. Hence the singleton $\{\omega \}$ for a fixed path $\omega$ is generally not in ${\mathcal{E}}_T$, and sets like "the path is everywhere continuous" need not be measurable. This is why path-regularity is a separate construction (a modification or a measure on a smaller path space), not automatic from the extension.

• Consistency is necessary but the topological hypothesis is also load-bearing. A consistent f.d.d. family need not extend to a countably additive measure if the coordinate spaces are arbitrary. The cylinder pre-measure ${\mu }_0$ is always finitely additive on $\mathcal{A}$; countable additivity (needed for Carathéodory 02.07.02) requires a regularity/inner-compactness input. The standard sufficient hypothesis is that each $E_t$ is a Polish space (or standard Borel). Andersen and Jessen exhibited consistent families on general spaces with no extension.

• Pairwise independence is strictly weaker than independence. Three events can be pairwise independent yet not mutually independent: on two fair coin flips let $A=\{$first H$\}$, $B=\{$second H$\}$, $C=\{$flips differ$\}$. Each pair is independent ($P=1/4$) but $P(A\cap B\cap C)=0\ne 1/8=P(A)P(B)P(C)$. Independence is a condition on the joint law of every finite subfamily, not just pairs.

• The law lives downstream; the space is interchangeable. Two different probability spaces can carry random variables with the same law. The pushforward $P_X$ records everything statistically observable about $X$; the underlying $(\Omega ,\mathcal{F},P)$ is scaffolding. This is why the canonical space $\Omega ={\prod }_t{E}_t$ with $X_t={\pi }_t$ is a universal model: any process is realized on it by the extension theorem.

Key theorem with proof Intermediate+

Theorem (Kolmogorov / Daniell extension; Kolmogorov 1933 Grundbegriffe §III; Daniell 1919 Ann. of Math. 20, 281). Let $T$ be an index set and let each $(E_t,{\mathcal{E}}_t)$ be a standard Borel space (a Borel subset of a Polish space with its Borel $\sigma$-algebra); for concreteness take each $E_t=\mathbb{R}$ with ${\mathcal{E}}_t=\mathcal{B}(\mathbb{R})$. Let $({\mu }_S{)}_{S\subseteq T\text{ finite}}$ be a consistent family of finite-dimensional distributions. Then there exists a unique probability measure $P$ on the cylinder $\sigma$-algebra $(\Omega ,{\mathcal{E}}_T)$, $\Omega ={\prod }_{t\in T}{E}_t$, such that $$({\pi }_S{)}_{\ast }P={\mu }_S\text{for every finite }S\subseteq T.$$ Equivalently, $P({\pi }_S^{-1}(B))={\mu }_S(B)$ for all finite $S$ and $B\in {\mathcal{E}}_S$.

Proof. Define the cylinder pre-measure ${\mu }_0$ on the algebra $\mathcal{A}$ of cylinder sets by ${\mu }_0({\pi }_S^{-1}(B))={\mu }_S(B)$.

Step 1 (well-definedness). A cylinder set has many representations: ${\pi }_S^{-1}(B)={\pi }_{S^{\prime }}^{-1}(B^{\prime })$ whenever $S\subseteq S^{\prime }$ and $B^{\prime }=p_{S^{\prime }S}^{-1}(B)$. Consistency gives ${\mu }_{S^{\prime }}(B^{\prime })={\mu }_{S^{\prime }}(p_{S^{\prime }S}^{-1}(B))=((p_{S^{\prime }S}{)}_{\ast }{\mu }_{S^{\prime }})(B)={\mu }_S(B)$, so ${\mu }_0$ does not depend on the representing base. For two arbitrary representations, pass to the common refinement $S\cup S^{\prime }$ and apply consistency to both. Hence ${\mu }_0$ is well-defined on $\mathcal{A}$, and ${\mu }_0(\Omega )={\mu }_{\varnothing }(E_{\varnothing })=1$.

Step 2 (finite additivity). If $C_1,C_2\in \mathcal{A}$ are disjoint, choose a finite base $S$ large enough to express both ($C_i={\pi }_S^{-1}(B_i)$ with $B_1,B_2$ disjoint, by enlarging bases as in Step 1). Then ${\mu }_0(C_1\bigsqcup C_2)={\mu }_S(B_1\bigsqcup B_2)={\mu }_S(B_1)+{\mu }_S(B_2)={\mu }_0(C_1)+{\mu }_0(C_2)$, using additivity of the single measure ${\mu }_S$.

Step 3 (countable additivity on the algebra — the compactness step). By the standard equivalence, a finitely additive probability ${\mu }_0$ on an algebra is countably additive if and only if it is continuous at $\varnothing$: for every decreasing sequence $C_1\supseteq C_2\supseteq \cdots$ in $\mathcal{A}$ with ${\bigcap }_n{C}_n=\varnothing$, one has ${\mu }_0(C_n)\to 0$. We prove the contrapositive: suppose ${\mu }_0(C_n)\ge \epsilon >0$ for all $n$; we show ${\bigcap }_n{C}_n\ne \varnothing$.

Each $C_n$ has a finite base; by enlarging we may take the bases to be an increasing sequence $S_1\subseteq S_2\subseteq \cdots$ with $C_n={\pi }_{S_n}^{-1}(B_n)$, $B_n\in {\mathcal{E}}_{S_n}$. Each $E_{S_n}$ is a Polish space, so its Borel probability ${\mu }_{S_n}$ is inner regular by compact sets (tightness on Polish spaces). Choose a compact $K_n\subseteq B_n$ with ${\mu }_{S_n}(B_n\setminus K_n)<\epsilon 2^{-n-1}$, and set $D_n={\pi }_{S_n}^{-1}(K_n)\subseteq C_n$. Replacing $D_n$ by $D_1\cap \cdots \cap D_n$ (still a compact-based cylinder, still inside $C_n$), the inclusion-exclusion estimate ${\mu }_0(C_n\setminus {\bigcap }_{k\le n}{D}_k)\le {\sum }_{k\le n}{\mu }_0(C_k\setminus D_k)<\epsilon /2$ gives ${\mu }_0({\bigcap }_{k\le n}{D}_k)>\epsilon /2>0$, so each ${\bigcap }_{k\le n}{D}_k$ is nonempty.

Pick ${\omega }^{(n)}\in {\bigcap }_{k\le n}{D}_k$. Its projection ${\pi }_{S_1}({\omega }^{(n)})$ lies in the compact $K_1$ for all $n$; pass to a subsequence converging in $K_1$. Diagonalize over the increasing bases $S_m$: by repeatedly extracting subsequences, obtain a single subsequence along which ${\pi }_{S_m}({\omega }^{(n)})$ converges in $K_m$ for every $m$. The limits are coordinatewise consistent and define a point ${\omega }^{\ast }\in \Omega$ with ${\pi }_{S_m}({\omega }^{\ast })\in K_m$ for all $m$, hence ${\omega }^{\ast }\in D_m\subseteq C_m$ for all $m$. Thus ${\bigcap }_n{C}_n\ne \varnothing$, the desired contradiction. So ${\mu }_0$ is countably additive on $\mathcal{A}$.

Step 4 (Carathéodory extension). A countably additive probability on the algebra $\mathcal{A}$ extends, by the Carathéodory extension theorem 02.07.02, to a measure $P$ on $\sigma (\mathcal{A})={\mathcal{E}}_T$. Since ${\mu }_0(\Omega )=1$, $P$ is a probability measure, and $P{|}_{\mathcal{A}}={\mu }_0$ gives $({\pi }_S{)}_{\ast }P={\mu }_S$ for every finite $S$.

Step 5 (uniqueness). The cylinder sets form a $\pi$-system generating ${\mathcal{E}}_T$, and any two probability measures agreeing on a generating $\pi$-system agree on the $\sigma$-algebra (Dynkin's $\pi$-$\lambda$ theorem). Two measures with the same f.d.d.'s agree on all cylinders, hence coincide. $\square$

Bridge. This theorem builds toward the entire probabilistic spine and appears again in every later construction of a stochastic process from its finite-dimensional laws. The foundational reason it is the right organizing tool is that it converts an analytically intractable object — a measure on an uncountable-dimensional path space — into the elementary data of consistent finite-dimensional marginals. This is exactly the move that constructs an i.i.d. sequence (take ${\mu }_S={⨂}_{t\in S}\nu$ for a fixed marginal $\nu$, automatically consistent by Fubini-Tonelli 02.07.07) and, more generally, any process once its f.d.d.'s are specified. The compactness argument of Step 3 generalises the tightness criterion that recurs in weak-convergence theory, and the Carathéodory backbone is exactly the one used to build Lebesgue and product measures 02.07.02, so putting these together the extension theorem is the probabilistic face of the same outer-measure machine. The central insight is that consistency of marginals plus a topological regularity hypothesis on the coordinate spaces is necessary and sufficient to manufacture a single coherent law, and this law is the canonical model on which the process realizes as the coordinate maps.

Exercises Intermediate+

Advanced results Master

The structure organizing the foundations splits into the canonical-space realization, the projective-limit formulation, the role of regularity, the regular-conditional-distribution refinement, and the special-case constructions (i.i.d. sequences, Markov chains via Ionescu-Tulcea, Gaussian processes) that the extension theorem subsumes or complements.

Theorem 1 (canonical realization). For any consistent f.d.d. family on standard Borel coordinates, the canonical space $(\Omega ,{\mathcal{E}}_T,P)=({\prod }_t{E}_t,{⨂}_t{\mathcal{E}}_t,P)$ with coordinate processes $X_t={\pi }_t$ realizes a process whose finite-dimensional distributions are exactly the prescribed ${\mu }_S$. Every process with those f.d.d.'s on any other space has the same law as the coordinate process under the law-isomorphism $\omega \mapsto (X_t(\omega ){)}_t$. The canonical space is therefore universal: it is the terminal object among realizations, and statistical statements transfer along the law map.

Theorem 2 (projective-limit formulation; Bochner). A consistent family $({\mu }_S)$ is a projective system of probability measures along the directed set of finite subsets ordered by inclusion, with bonding maps the marginal projections. The Kolmogorov measure $P$ is the projective limit $\underleftarrow{\lim }{\mu }_S$ in the category of probability spaces, characterized by $({\pi }_S{)}_{\ast }P={\mu }_S$ together with the universal property that any cone of measure-preserving maps factors uniquely through $P$. Bochner's theorem on projective limits gives existence under the same regularity hypothesis; the cylinder-algebra proof and the projective-limit proof are two presentations of one construction.

Theorem 3 (Nelson regularity / inner-compact criterion; Nelson 1959 Ann. of Math. 69, 630). Countable additivity of the cylinder pre-measure holds whenever each ${\mu }_S$ is inner regular by compact sets in a compatible topology on $E_S$ that is compatible across marginalization. Polish coordinate spaces satisfy this; more generally, a perfect family or a Radon family suffices. The criterion isolates the exact analytic input: the algebraic data (consistency) is never enough on its own — a topological tightness ingredient is required, and it is sharp in the sense of the Dieudonné counterexample.

Theorem 4 (regular conditional distributions and disintegration). On standard Borel spaces, for a random variable $X$ and a sub-$\sigma$-algebra $\mathcal{G}$, there is a regular conditional distribution: a kernel $\kappa :\Omega \times \mathcal{E}\to [0,1]$ with $\kappa (\omega ,\cdot )$ a probability measure and $\kappa (\cdot ,B)$ a version of $P(X\in B\mid \mathcal{G})$. The disintegration theorem factors a joint law into a marginal and a measurable family of conditionals, $\mu (dxdy)={\mu }_1(dx)\kappa (x,dy)$. This refines the product construction: independence is the special case $\kappa (x,\cdot )={\mu }_2$ constant in $x$. Standard Borel structure is exactly what makes such regular versions exist.

Theorem 5 (Ionescu-Tulcea: extension without topology for kernels). If instead of a consistent family one is given a sequence of probability kernels ${\kappa }_n:E_1\times \cdots \times E_n⇝E_{n+1}$ and an initial law ${\nu }_1$, the Ionescu-Tulcea theorem constructs a measure on ${\prod }_n{E}_n$ with the prescribed conditionals, requiring no topological hypothesis on the $E_n$. This is the natural tool for Markov chains and for any sequential model defined by transition kernels. The Kolmogorov theorem and the Ionescu-Tulcea theorem are complementary: Kolmogorov needs consistency plus topology and handles arbitrary index sets; Ionescu-Tulcea needs a kernel sequence (an ordered index) and dispenses with topology.

Synthesis. The foundational reason the probability spine can be built at all is that the Kolmogorov extension theorem reduces the construction of a measure on an uncountable-dimensional space to consistent finite-dimensional data, and this is exactly the move that recurs at every level above it. Putting these together, the canonical-space realization (Theorem 1) makes the coordinate process the universal model, the projective-limit formulation (Theorem 2) is dual to the cylinder-algebra construction and exhibits $P$ as $\underleftarrow{\lim }{\mu }_S$, and the regularity criterion (Theorem 3) isolates the load-bearing tightness input that the algebraic consistency cannot supply. The central insight is that consistency generalises the Fubini-Tonelli product structure 02.07.07: an i.i.d. law is the product family, a Markov chain is the kernel-composed family, and a Gaussian process is the family of multivariate normals with a consistent covariance — each a different consistent system fed to the same machine. The bridge is between the outer-measure Carathéodory backbone 02.07.02 and the entire theory of stochastic processes: the extension theorem appears again in the construction of Brownian motion (where Kolmogorov supplies the law on ${\mathbb{R}}^{[0,\infty )}$ and a separate continuity theorem selects a continuous modification), in the strong law and the ergodic theorem (which live on the canonical i.i.d. space), and in the disintegration theory (Theorem 4) that conditions the canonical law. This is the structural fact that organises the foundations: every later existence result is a specialization, and the regularity-versus-kernels dichotomy (Kolmogorov versus Ionescu-Tulcea) is the recurring decision about which hypothesis to pay for.

Full proof set Master

Proposition 1 (countable additivity equals continuity at $\varnothing$). Let ${\mu }_0$ be a finitely additive probability content on an algebra $\mathcal{A}$. Then ${\mu }_0$ is countably additive on $\mathcal{A}$ (in the sense that ${⨆}_n{C}_n=C\in \mathcal{A}$ with $C_n\in \mathcal{A}$ disjoint implies ${\mu }_0(C)={\sum }_n{\mu }_0(C_n)$) if and only if $C_n\downarrow \varnothing$ in $\mathcal{A}$ implies ${\mu }_0(C_n)\to 0$.

Proof. ($\Rightarrow$) If $C_n\downarrow \varnothing$, the differences $D_n=C_n\setminus C_{n+1}$ are disjoint with $C_m={⨆}_{n\ge m}{D}_n$ (union in $\mathcal{A}$ as a tail). Countable additivity gives ${\mu }_0(C_1)={\sum }_{n\ge 1}{\mu }_0(D_n)<\infty$, so the tail ${\mu }_0(C_m)={\sum }_{n\ge m}{\mu }_0(D_n)\to 0$. ($\Leftarrow$) Suppose continuity at $\varnothing$. Let $C={⨆}_n{C}_n\in \mathcal{A}$ with $C_n$ disjoint in $\mathcal{A}$. The remainders $R_N=C\setminus {⨆}_{n\le N}{C}_n$ lie in $\mathcal{A}$, decrease to $\varnothing$, so ${\mu }_0(R_N)\to 0$. Finite additivity gives ${\mu }_0(C)={\sum }_{n\le N}{\mu }_0(C_n)+{\mu }_0(R_N)$; letting $N\to \infty$ yields ${\mu }_0(C)={\sum }_n{\mu }_0(C_n)$. $\square$

Proposition 2 (well-definedness of the cylinder pre-measure). Under consistency, ${\mu }_0({\pi }_S^{-1}(B))={\mu }_S(B)$ is independent of the chosen base representation.

Proof. Suppose ${\pi }_S^{-1}(B)={\pi }_{S^{\prime }}^{-1}(B^{\prime })$. Let $U=S\cup S^{\prime }$ with projections $q:E_U\to E_S$, $q^{\prime }:E_U\to E_{S^{\prime }}$. Pulling back, $q^{-1}(B)=q^{\prime -1}(B^{\prime })$ as subsets of $E_U$ (both equal ${\pi }_U({\pi }_S^{-1}(B))$'s base). Consistency gives ${\mu }_S(B)=(q_{\ast }{\mu }_U)(B)={\mu }_U(q^{-1}(B))={\mu }_U(q^{\prime -1}(B^{\prime }))=(q_{\ast }^{\prime }{\mu }_U)(B^{\prime })={\mu }_{S^{\prime }}(B^{\prime })$. Hence the two candidate values coincide. $\square$

Proposition 3 (tightness on Polish spaces). Every Borel probability measure $\mu$ on a Polish space $E$ is inner regular by compact sets: $\mu (B)=\sup \{\mu (K):K\subseteq B\text{ compact}\}$ for every Borel $B$.

Proof. Fix a complete separable metric. For $\epsilon >0$ and each $n$, separability gives countably many closed balls of radius $1/n$ covering $E$; finitely many of them, say with union $F_n$, satisfy $\mu (F_n)>1-\epsilon 2^{-n}$. The set $K={\bigcap }_n{F}_n$ is closed and totally bounded, hence compact by completeness, with $\mu (E\setminus K)\le {\sum }_n\epsilon 2^{-n}=\epsilon$. Thus $\mu (K)>1-\epsilon$. Inner regularity of Borel sets then follows from outer/inner regularity by closed/open sets (standard for metric measures) intersected with this compact exhaustion. $\square$

Proposition 4 (the compactness/diagonal step). With the notation of Step 3 of the Key theorem, if ${\mu }_0(C_n)\ge \epsilon$ for all $n$ then ${\bigcap }_n{C}_n\ne \varnothing$.

Proof. As constructed, $D_n={\bigcap }_{k\le n}{\pi }_{S_k}^{-1}(K_k)$ are nonempty compact-based cylinders with $D_n\subseteq C_n$ and $D_{n+1}\subseteq D_n$. Pick ${\omega }^{(n)}\in D_n$. The points ${\pi }_{S_1}({\omega }^{(n)})\in K_1$ (compact) admit a convergent subsequence; inductively refine so that for each fixed $m$, ${\pi }_{S_m}({\omega }^{(n)})$ converges in $K_m$ along a single diagonal subsequence. The limits $a_m\in K_m$ are consistent under the projections $E_{S_{m+1}}\to E_{S_m}$ (projections are continuous, limits commute), so they assemble into ${\omega }^{\ast }\in \Omega$ with ${\pi }_{S_m}({\omega }^{\ast })=a_m\in K_m$. Then ${\omega }^{\ast }\in {\pi }_{S_m}^{-1}(K_m)\supseteq D_m$-defining sets, giving ${\omega }^{\ast }\in C_m$ for every $m$, so ${\omega }^{\ast }\in {\bigcap }_m{C}_m$. $\square$

Proposition 5 (uniqueness via $\pi$-$\lambda$). A probability measure on ${\mathcal{E}}_T$ is determined by its values on cylinders.

Proof. As in Exercise 7: cylinders form a $\pi$-system generating ${\mathcal{E}}_T$; the agreement set of two probability measures is a $\lambda$-system; Dynkin's theorem forces agreement on $\sigma$(cylinders) $={\mathcal{E}}_T$. $\square$

Proposition 6 (pushforward is a probability measure). If $X:(\Omega ,\mathcal{F},P)\to (E,\mathcal{E})$ is measurable then $P_X=P\circ X^{-1}$ is a probability measure on $(E,\mathcal{E})$.

Proof. $P_X(E)=P(X^{-1}(E))=P(\Omega )=1$. For disjoint $B_n\in \mathcal{E}$, the preimages $X^{-1}(B_n)$ are disjoint in $\mathcal{F}$ and $X^{-1}({⨆}_n{B}_n)={⨆}_n{X}^{-1}(B_n)$, so countable additivity of $P$ transfers: $P_X({⨆}_n{B}_n)=P({⨆}_n{X}^{-1}(B_n))={\sum }_{n}P(X^{-1}(B_n))={\sum }_n{P}_X(B_n)$. Non-negativity is inherited. $\square$

Proposition 7 (consistency of i.i.d. and of Gaussian families). The product family ${\mu }_S={⨂}_{t\in S}\nu$ is consistent (Exercise 3). The Gaussian family ${\mu }_S=\mathcal{N}(0,{\Sigma }_S)$ with ${\Sigma }_S=(c(s,t){)}_{s,t\in S}$, for a symmetric positive-semidefinite kernel $c:T\times T\to \mathbb{R}$, is consistent.

Proof. For the Gaussian case, marginalizing a multivariate normal $\mathcal{N}(0,{\Sigma }_{S^{\prime }})$ onto the coordinates in $S\subseteq S^{\prime }$ yields $\mathcal{N}(0,{\Sigma }_S)$, because the marginal of a jointly Gaussian vector is Gaussian with the corresponding sub-covariance block (the projection of a normal law is normal, with covariance the principal submatrix). Positive-semidefiniteness of $c$ guarantees each ${\Sigma }_S$ is a valid covariance, so each ${\mu }_S$ exists. Marginal-consistency is the sub-block identity, so the family is projective and the extension yields a centered Gaussian process with covariance $c$. $\square$

Connections Master

• The product-measure and Fubini-Tonelli theory 02.07.07 is the finite-dimensional engine of consistency: an i.i.d. family is the projective system of product measures, and marginalizing a product is precisely a Tonelli integration over the dropped coordinates. The extension theorem is the infinite-index continuation of the finite product construction, so this unit generalises 02.07.07 from finite to arbitrary index sets.

• The Carathéodory extension and outer-measure construction 02.07.02 supplies the backbone of Step 4: the cylinder pre-measure, once shown countably additive on the algebra, is extended to the generated $\sigma$-algebra by the same outer-measure machine that builds Lebesgue measure. The probabilistic content is entirely in establishing countable additivity; the lift is borrowed verbatim from 02.07.02.

• The $\sigma$-algebra and Borel-space foundations 02.07.01 define the cylinder $\sigma$-algebra and the standard-Borel hypothesis that makes the regularity step work; measurability of the coordinate maps and of random variables is exactly the framework of 02.07.01, and the countable-coordinate-dependence phenomenon is a structural fact about product $\sigma$-algebras introduced there.

• The elementary rules and distributions of probability 26.02.01 are the concrete shadow of this unit: the axioms $P(\Omega )=1$ and additivity, independence as a product rule, and named distributions all reappear here as the measure-theoretic objects they abstract, with 26.02.01's finite and discrete cases embedded as the f.d.d.'s the extension theorem stitches together.

Historical & philosophical context Master

The measure-theoretic foundation of probability was settled by Andrei Kolmogorov's 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ^{[Kolmogorov 1933]}, which identified a random experiment with a measure space of total mass one and a random variable with a measurable function. The extension theorem occupies the final chapter (§III), where Kolmogorov constructs measures on infinite-dimensional product spaces from consistent finite-dimensional distributions, the existence result that legitimized the study of stochastic processes as honest mathematical objects rather than heuristic limits.

The analytic heart of the construction predates Kolmogorov. Percy Daniell's 1919 paper Integrals in an infinite number of dimensions ^{[Daniell 1919]} built integration on infinite products and proved the extension for product-type families, and the result is often called the Daniell-Kolmogorov theorem in recognition. The Carathéodory extension machinery it relies on came from Constantin Carathéodory's 1918 Vorlesungen über reelle Funktionen ^{[Caratheodory 1918]}, the outer-measure axiomatization used identically for Lebesgue measure.

The topological hypothesis was clarified over the following decades. Jean Dieudonné (1948) and the work surveyed by Erik Sparre Andersen and Børge Jessen produced consistent families on non-regular coordinate spaces with no countably additive extension, showing that consistency alone is insufficient. Edward Nelson's 1959 Regular probability measures on function space ^{[Nelson 1959]} gave a clean inner-compactness criterion, and K. R. Parthasarathy's 1967 Probability Measures on Metric Spaces ^{[Parthasarathy 1967]} established the theorem in its standard Polish-space form, with the Ionescu-Tulcea kernel construction providing the complementary topology-free route for sequentially defined models.

Bibliography Master

@book{Kolmogorov1933,
  author    = {Kolmogorov, Andrei N.},
  title     = {Grundbegriffe der {W}ahrscheinlichkeitsrechnung},
  series    = {Ergebnisse der Mathematik und ihrer Grenzgebiete},
  publisher = {Springer},
  address   = {Berlin},
  year      = {1933}
}

@article{Daniell1919,
  author  = {Daniell, Percy J.},
  title   = {Integrals in an infinite number of dimensions},
  journal = {Annals of Mathematics (2)},
  volume  = {20},
  year    = {1919},
  pages   = {281--288}
}

@book{Caratheodory1918,
  author    = {Carath\'eodory, Constantin},
  title     = {Vorlesungen \"uber reelle {F}unktionen},
  publisher = {Teubner},
  address   = {Leipzig and Berlin},
  year      = {1918}
}

@article{Nelson1959,
  author  = {Nelson, Edward},
  title   = {Regular probability measures on function space},
  journal = {Annals of Mathematics (2)},
  volume  = {69},
  year    = {1959},
  pages   = {630--643}
}

@book{Parthasarathy1967,
  author    = {Parthasarathy, K. R.},
  title     = {Probability Measures on Metric Spaces},
  publisher = {Academic Press},
  address   = {New York},
  year      = {1967}
}

@book{Durrett2019,
  author    = {Durrett, Rick},
  title     = {Probability: Theory and Examples},
  edition   = {5},
  publisher = {Cambridge University Press},
  year      = {2019}
}

@book{Kallenberg2021,
  author    = {Kallenberg, Olav},
  title     = {Foundations of Modern Probability},
  edition   = {3},
  publisher = {Springer},
  year      = {2021}
}

@book{Billingsley1995,
  author    = {Billingsley, Patrick},
  title     = {Probability and Measure},
  edition   = {3},
  publisher = {Wiley},
  year      = {1995}
}

@book{Bogachev2007v2,
  author    = {Bogachev, Vladimir I.},
  title     = {Measure Theory, Volume 2},
  publisher = {Springer},
  year      = {2007}
}