Kolmogorov-Sinai Entropy and the Generator Theorem

Intuition Beginner

Imagine watching a machine that shuffles a deck and then shows you only one coarse fact about the result each round — say, whether the top card is red or black. How much can you learn about the machine by watching forever? Some machines are tame: after a few rounds you can predict every future report perfectly, so the stream of reports stops surprising you. Others are wild: no matter how long you watch, each new report still carries genuinely fresh information you could not have guessed. Entropy is the single number that measures exactly this — the average amount of new surprise the system produces per step over the long run.

The trick is to turn "surprise" into arithmetic. Group all possible states of the system into a handful of labelled bins. At each step the system lands in some bin, producing a label. Watching for $n$ steps gives a string of $n$ labels, like a word in a small alphabet. A predictable system produces only a few possible words of each length; a chaotic one produces exponentially many. Entropy is the growth rate of that count: the rate at which the number of distinguishable length-$n$ histories multiplies. A system whose histories double in number every step has entropy equal to the logarithm of $2$.

You measure the system through the coarse bins you chose, so a single choice of bins gives only a lower estimate of the true complexity. The full entropy is what you get by choosing the best bins — the finest practical way of looking. A remarkable shortcut, the generator theorem, says you do not have to hunt forever: if your bins are rich enough that watching the entire infinite future of labels pins down the exact starting state, then those bins already see all the entropy there is. One good measurement scheme, watched forever, reveals everything.

The takeaway: entropy counts the average new information per step, equivalently the exponential growth rate of distinguishable histories. It is the sharpest number telling apart a clockwork system from a genuinely unpredictable one, and it stays the same under any relabelling that preserves the dynamics, which is what makes it a true fingerprint of the system.

Visual Beginner

Picture a binary tree of histories growing one level per time step. At the root you know nothing; each step the system reveals one new label and you descend one level, splitting into branches.

The full tree on the left doubles at every level, so it has $2^n$ histories of length $n$ and entropy equal to the logarithm of $2$. The starved tree on the right barely grows: few histories, so almost no surprise per step and entropy near zero. The table shows that two very different-looking systems — a fair coin and the doubling map — share the same entropy, $\log 2$, because both reveal one fresh bit per step.

Worked example Beginner

We compute the entropy of the simplest random source: a biased coin flipped forever, with bins "heads" and "tails".

Step 1. The setup. Each step the system reports H or T independently, with the chance of H equal to $p=0.25$ and the chance of T equal to $0.75$. The two bins are "the report is H" and "the report is T". A length-$n$ history is a string like HTTTH of H's and T's.

Step 2. Surprise of one report. Information theory measures the surprise of an outcome of chance $q$ as $\log (1/q)$, using logarithm base $2$ so the answer comes out in bits. An H, of chance $0.25$, carries $\log (1/0.25)=\log 4=2$ bits of surprise. A T, of chance $0.75$, carries $\log (1/0.75)=\log (1.333)=0.415$ bits.

Step 3. Average surprise per step. Average the two surprises weighted by how often each occurs: $0.25\times 2+0.75\times 0.415=0.5+0.311=0.811$ bits. This weighted average of the per-report surprises is the entropy of one step.

Step 4. The long-run rate. Because the reports are independent, watching $n$ steps gives exactly $n$ times the one-step surprise on average, so the entropy of the whole system is $0.811$ bits per step. A fair coin ($p=0.5$) would instead give $0.5\times 1+0.5\times 1=1$ bit per step, the maximum for two bins.

What this tells us: the biased coin produces about $0.811$ bits of genuine novelty per flip, less than the fair coin's full $1$ bit because the bias makes T somewhat predictable. The entropy formula simply averages the surprise of each outcome by its probability — small probabilities are very surprising but rare, large probabilities are unsurprising but common, and entropy balances the two.

Check your understanding Beginner

Formal definition Intermediate+

Throughout, $(X,\mathcal{B},\mu ,T)$ is a measure-preserving system 38.04.01 on a probability space, and all logarithms are natural unless a base is named. A finite measurable partition $P=\{P_1,\dots ,P_k\}$ of $X$ is a finite collection of pairwise-disjoint measurable sets with $\mu ({\bigcup }_i{P}_i)=1$.

Definition (entropy of a partition). The entropy of a finite partition $P$ is $$H(P)=-\sum _{i=1}^k\mu (P_i)\log \mu (P_i),$$ with the convention $0\log 0=0$. Writing $\phi (t)=-t\log t$ for $t\in [0,1]$ (continuous, concave, $\phi (0)=\phi (1)=0$), $H(P)={\sum }_i\phi (\mu (P_i))$. By concavity of $\phi$, $0\le H(P)\le \log k$, with the maximum $\log k$ attained exactly when all atoms have equal measure $1/k$.

Definition (join and refinement). The join (common refinement) of partitions $P$ and $Q$ is $P\vee Q=\{P_i\cap Q_j:\mu (P_i\cap Q_j)>0\}$. We write $P⪯Q$ ($Q$ refines $P$) when every atom of $Q$ lies in some atom of $P$. For the transformation $T$, $T^{-1}P=\{T^{-1}{P}_i\}$ is again a partition with $\mu (T^{-1}{P}_i)=\mu (P_i)$, so $H(T^{-1}P)=H(P)$.

Definition (conditional entropy). The conditional entropy of $P$ given $Q$ is $$H(P\mid Q)=-\sum _{i,j}\mu (P_i\cap Q_j)\log \frac{\mu (P_i\cap Q_j)}{\mu (Q_j)}=\sum _j\mu (Q_j)H(P\mid Q_j),$$ the average over atoms of $Q$ of the entropy of $P$ restricted to that atom. It satisfies the chain rule $H(P\vee Q)=H(Q)+H(P\mid Q)$ and the monotonicity $H(P\mid Q)\le H(P)$, with equality iff $P$ and $Q$ are independent; refining the conditioning partition decreases conditional entropy, $Q⪯Q^{\prime }\Rightarrow H(P\mid Q^{\prime })\le H(P\mid Q)$.

Definition (entropy of a partition relative to $T$). Set $P_0^{n-1}={\bigvee }_{k=0}^{n-1}{T}^{-k}P$, the partition by the first $n$ symbols of the $P$-itinerary. The sequence $a_n=H(P_0^{n-1})$ is subadditive: $a_{m+n}\le a_m+a_n$. The entropy of $P$ relative to $T$ is the limit $$h(T,P)=\underset{n\to \infty }{\lim }\frac{1}{n}H\left(\underset{k=0}{\overset{n-1}{\bigvee }}{T}^{-k}P\right)=\underset{n\ge 1}{\inf }\frac{1}{n}H\left(\underset{k=0}{\overset{n-1}{\bigvee }}{T}^{-k}P\right),$$ existing by Fekete's subadditivity lemma 37.02.03. Equivalently $h(T,P)={\lim }_{n}H(P\mid {\bigvee }_{k=1}^n{T}^{-k}P)$, the asymptotic uncertainty in the present symbol given the entire past.

Definition (Kolmogorov-Sinai entropy). The measure-theoretic (Kolmogorov-Sinai) entropy of $(X,\mathcal{B},\mu ,T)$ is $$h(T)=h_{\mu }(T)=\underset{P}{\sup }h(T,P),$$ the supremum over all finite measurable partitions $P$. It is a measurable-conjugacy invariant: if $\Phi :X\to Y$ is a measure-isomorphism intertwining $T$ and $S$ (i.e. $\Phi \circ T=S\circ \Phi$ a.e.), then $h_{\mu }(T)=h_{\nu }(S)$.

Definition (generator). A finite partition $P$ is a generator for an invertible $T$ if ${\bigvee }_{k=-\infty }^{\infty }{T}^{-k}P=\mathcal{B}$ modulo $\mu$-null sets — the symbolic itinerary determines the point a.e. For non-invertible $T$ the corresponding notion uses one-sided refinements ${\bigvee }_{k=0}^{\infty }{T}^{-k}P$.

Counterexamples to common slips Intermediate+

• Entropy of a partition is not entropy of the system. $H(P)$ measures one partition's static information; $h(T,P)$ measures its dynamical growth rate; $h(T)$ takes the supremum. The doubling map with the one-atom partition $P=\{X\}$ has $h(T,P)=0$ even though $h(T)=\log 2$. A single bad partition sees nothing.

• The supremum is not attained by an arbitrary partition. $h(T)={\sup }_{P}h(T,P)$ genuinely requires either a generator or a refining sequence; computing $h(T,P)$ for one convenient $P$ gives only a lower bound. The generator theorem is precisely the tool that replaces the supremum by a single evaluation when $P$ generates.

• Conditional entropy decreases the integrand, not always the join entropy. $H(P\mid Q)\le H(P)$ always, but $H(P\vee Q)=H(Q)+H(P\mid Q)\ge H(Q)$: refining never lowers static entropy. Confusing "conditioning lowers entropy" with "joining lowers entropy" reverses an inequality.

• Measure-theoretic entropy is not topological entropy. The doubling map has measure entropy $\log 2$ for Lebesgue measure but its topological entropy is also $\log 2$ only because Lebesgue is the measure of maximal entropy; for a different invariant measure $h_{\mu }(T)$ can be strictly smaller. The variational principle $h_{\mathrm{top}}(T)={\sup }_{\mu }{h}_{\mu }(T)$ relates them but does not identify them.

• Zero entropy is not the same as periodicity or non-ergodicity. An irrational rotation is ergodic, aperiodic, and has entropy $0$: it is deterministic in the entropy sense (the past determines the present exactly) yet equidistributes. Entropy measures unpredictability, a finer distinction than the recurrence and ergodicity of 38.04.01.

Key theorem with proof Intermediate+

Theorem (Kolmogorov-Sinai generator theorem; Kolmogorov 1958, Sinai 1959). Let $(X,\mathcal{B},\mu ,T)$ be an invertible measure-preserving system and let $P$ be a finite partition with $H(P)<\infty$ that is a generator, meaning ${\bigvee }_{k=-\infty }^{\infty }{T}^{-k}P=\mathcal{B}(\mathrm{mod}\mu )$. Then $$h(T)=h(T,P).$$ The supremum defining the entropy is attained at any generating partition, so a single computation of $h(T,P)$ yields the full invariant.

Proof. Since $h(T)={\sup }_{Q}h(T,Q)\ge h(T,P)$, it suffices to show $h(T,Q)\le h(T,P)$ for every finite partition $Q$. The engine is the inequality $$h(T,Q)\le h(T,P)+H(Q\mid P_{-m}^m),P_{-m}^m=\underset{k=-m}{\overset{m}{\bigvee }}{T}^{-k}P,\text{\hspace{1em}(∗)}$$ combined with $H(Q\mid P_{-m}^m)\to 0$ as $m\to \infty$, which is where the generator hypothesis enters.

First, the generator hypothesis gives the limit. The partitions $P_{-m}^m$ increase to the $\sigma$-algebra they generate, which is ${\bigvee }_{k\in \mathbb{Z}}{T}^{-k}P=\mathcal{B}(\mathrm{mod}\mu )$. Conditional entropy of a fixed finite partition given an increasing sequence of partitions converges to the conditional entropy given the limit $\sigma$-algebra: $H(Q\mid P_{-m}^m)\to H(Q\mid \mathcal{B})=0$, since $Q\subseteq \mathcal{B}$ makes $Q$ measurable with respect to the conditioning $\sigma$-algebra in the limit. This uses the increasing-martingale convergence of conditional expectations 37.02.03 applied to the indicator functions $1_{Q_j}$.

Now establish $(\ast )$. By the chain rule and subadditivity of $h(T,\cdot )$ in its partition argument, for any partitions $Q$ and $R$, $$h(T,Q)\le h(T,R)+H(Q\mid R).$$ Indeed $h(T,Q)\le h(T,Q\vee R)$ and $h(T,Q\vee R)\le h(T,R)+h(T,Q\mid R)\le h(T,R)+H(Q\mid R)$, the last step because the relative entropy $h(T,Q\mid R)={\lim }_n\frac{1}{n}H(Q_0^{n-1}\mid R_0^{n-1})\le H(Q\mid R)$ by subadditivity of conditional entropy along the refinement. Take $R=P_{-m}^m$. Because $T^{-1}{P}_{-m}^m=P_{-m-1}^{m-1}$ differs from $P_{-m}^m$ only by a shift of the index window, $h(T,P_{-m}^m)=h(T,P)$: the entropy of a transformation is unchanged when the partition is replaced by a finite join of its own iterates, since $\frac{1}{n}H\left({\bigvee }_{k=0}^{n-1}{T}^{-k}{P}_{-m}^m\right)=\frac{1}{n}H\left({\bigvee }_{k=-m}^{n-1+m}{T}^{-k}P\right)=\frac{n+2m}{n}\cdot \frac{1}{n+2m}H(P_{-m}^{n-1+m})\to h(T,P)$. Substituting into the subadditivity inequality gives $(\ast )$: $$h(T,Q)\le h(T,P_{-m}^m)+H(Q\mid P_{-m}^m)=h(T,P)+H(Q\mid P_{-m}^m).$$ Letting $m\to \infty$, the conditional-entropy term vanishes, leaving $h(T,Q)\le h(T,P)$. Taking the supremum over $Q$ yields $h(T)\le h(T,P)$, and the reverse inequality is immediate, so $h(T)=h(T,P)$. $\square$

Bridge. The generator theorem builds toward every concrete entropy computation and appears again in the Bernoulli and toral-automorphism evaluations of the Advanced results, where the natural state partition is shown to generate and so its single relative entropy is the whole invariant. The foundational reason the supremum collapses is that a generating partition's iterates exhaust the $\sigma$-algebra, so no other partition can carry information invisible to $P$ — this is exactly the martingale convergence $H(Q\mid P_{-m}^m)\to 0$ read as "the past and future of $P$ eventually determine $Q$". Putting these together with 37.02.03, entropy is the dynamical analogue of the asymptotic equipartition property: the Shannon-McMillan-Breiman theorem says $-\frac{1}{n}\log \mu (P_0^{n-1}(x))\to h(T,P)$ almost everywhere, which is the central insight that a typical length-$n$ name has measure $e^{-nh(T,P)}$, so there are about $e^{nh(T,P)}$ typical names — the count whose growth rate the generator theorem certifies is the true entropy. The construction is dual to the recurrence-and-return picture of 38.04.01: where Kac counts return times, entropy counts the exponential proliferation of distinguishable orbits, and the Rokhlin tower that organised return times reappears as the technical device estimating $h(T,P)$ by cutting the dynamics into finite columns.

Exercises Intermediate+

Advanced results Master

Theorem 1 (existence and invariance of entropy; Kolmogorov 1958, Sinai 1959). For every measure-preserving system, $h(T,P)={\lim }_n\frac{1}{n}H(P_0^{n-1})$ exists for each finite $P$, and $h(T)={\sup }_{P}h(T,P)$ is invariant under measurable conjugacy. Moreover $h(T^m)=mh(T)$ for $m\ge 1$, and for invertible $T$, $h(T^{-1})=h(T)$ and $h(T^m)=|m|h(T)$ for $m\in \mathbb{Z}$. The Kolmogorov-Sinai invariant separated systems that the prior spectral invariants of 38.04.01 could not, most decisively the Bernoulli shifts ^{[Kolmogorov 1958]}.

Theorem 2 (generator theorem; Sinai 1959). If $P$ is a finite generating partition for an invertible $T$ with $H(P)<\infty$, then $h(T)=h(T,P)$. A countable generator with $H(P)<\infty$ suffices. The theorem reduces the entropy to one evaluation and is what makes entropy computable: every model below is an application ^{[Sinai 1959]}.

Theorem 3 (model computations). The Bernoulli shift $B(p_0,\dots ,p_{k-1})$ has $h=-{\sum }_i{p}_i\log p_i$; the doubling map $x\mapsto 2x\mathrm{mod}1$ has $h=\log 2$; more generally $x\mapsto mx\mathrm{mod}1$ has $h=\log m$; a hyperbolic toral automorphism $T_A$ has $h={\sum }_{|{\lambda }_i|>1}\log |{\lambda }_i|$, the sum of logarithms of the eigenvalues of modulus exceeding one. The last is the abelian case of the Pesin entropy formula $h_{\mu }(T)=\int {\sum }_i{\lambda }_i^{+}d\mu$ equating entropy with integrated positive Lyapunov exponents for smooth systems preserving a smooth measure ^{[Walters Ch. 4]}.

Theorem 4 (Kolmogorov's solution to the isomorphism problem for Bernoulli shifts). Entropy is a complete numerical obstruction within the Bernoulli class: in one direction equal entropy is necessary for isomorphism, and Ornstein's theorem supplies the converse: two Bernoulli shifts are measurably isomorphic if and only if they have the same entropy. Thus $B(1/2,1/2)$ and $B(1/3,1/3,1/3)$ are non-isomorphic ($\log 2\ne \log 3$), settling a question open since von Neumann, while $B(1/4,1/4,1/4,1/4)$ and $B(1/2,1/2)\times B(1/2,1/2)$ are isomorphic (both entropy $\log 4$) ^{[Ornstein 1970]}.

Theorem 5 (Shannon-McMillan-Breiman; the asymptotic equipartition property). For an ergodic system and a finite partition $P$ with $H(P)<\infty$, $-\frac{1}{n}\log \mu (P_0^{n-1}(x))\to h(T,P)$ for $\mu$-a.e. $x$ and in $L^1$, where $P_0^{n-1}(x)$ is the atom containing $x$. The number of $P$-names of length $n$ needed to carry all but $\epsilon$ of the measure grows like $e^{nh(T,P)}$, and each typical name has measure about $e^{-nh(T,P)}$. This is the dynamical entropy realised as an almost-sure growth rate, the bridge to coding and to the Pinsker $\sigma$-algebra of zero-entropy factors ^{[Walters Ch. 4]}.

Synthesis. The five results are one architecture, and the foundational reason they cohere is that the static partition entropy $H(P)$, once iterated under $T$ and normalised, becomes a growth rate that no relabelling can see. The generator theorem is exactly the statement that this growth rate is computed once and for all on any partition whose iterates exhaust the $\sigma$-algebra, and this is dual to the recurrence-and-return geometry of 38.04.01: where Kac's skyscraper counted return times, the entropy join counts the exponential proliferation of names, and the Rokhlin tower is the shared technical device. Putting these together with the Shannon-McMillan-Breiman theorem, the central insight is that a typical orbit-name of length $n$ has measure $e^{-nh}$, so $h$ is simultaneously an information rate, a counting exponent, and — by the Pesin formula — an integral of positive Lyapunov exponents; this is the bridge from the abstract measure-preserving system to smooth chaos. Kolmogorov's invariant generalises the spectral invariants that preceded it and resolves the Bernoulli isomorphism problem that those invariants could not touch, and Ornstein's converse shows entropy is not merely an obstruction but a complete one inside the Bernoulli class — the foundational reason entropy occupies the centre of the modern theory.

Full proof set Master

Proposition 1 (concavity bound $0\le H(P)\le \log k$). For a partition into $k$ atoms, $0\le H(P)\le \log k$, with the upper bound attained iff every atom has measure $1/k$.

Proof. Non-negativity is immediate since each term $\phi (\mu (P_i))=-\mu (P_i)\log \mu (P_i)\ge 0$ for $\mu (P_i)\in [0,1]$. For the upper bound, $\phi$ is concave, so by Jensen's inequality applied with uniform weights $1/k$, $$\frac{1}{k}\sum _{i=1}^k\phi (\mu (P_i))\le \phi (\frac{1}{k}\sum _i\mu (P_i))=\phi (1/k)=\frac{1}{k}\log k.$$ Multiplying by $k$ gives $H(P)={\sum }_i\phi (\mu (P_i))\le \log k$. Strict concavity of $\phi$ forces equality only when all $\mu (P_i)$ are equal, i.e. each equals $1/k$. $\square$

Proposition 2 (subadditivity of $a_n$ and existence of $h(T,P)$). $a_n=H(P_0^{n-1})$ satisfies $a_{m+n}\le a_m+a_n$, hence $a_n/n\to {\inf }_n{a}_n/n=h(T,P)$.

Proof. Write $R_0^{n-1}={\bigvee }_{k=0}^{n-1}{T}^{-k}P$. Then $P_0^{m+n-1}=P_0^{m-1}\vee T^{-m}{P}_0^{n-1}$. Subadditivity of partition entropy, $H(A\vee B)\le H(A)+H(B)$ — itself the chain rule $H(A\vee B)=H(B)+H(A\mid B)$ with $H(A\mid B)\le H(A)$ (Exercise 6) — gives $a_{m+n}\le H(P_0^{m-1})+H(T^{-m}{P}_0^{n-1})$. Measure-preservation yields $H(T^{-m}{P}_0^{n-1})=H(P_0^{n-1})=a_n$, so $a_{m+n}\le a_m+a_n$. Fekete's subadditivity lemma 37.02.03 gives $a_n/n\to {\inf }_n{a}_n/n$. $\square$

Proposition 3 (conditional form of $h(T,P)$). $h(T,P)={\lim }_{n}H(P\mid {\bigvee }_{k=1}^n{T}^{-k}P)=H(P\mid {\bigvee }_{k\ge 1}{T}^{-k}P)$.

Proof. By the chain rule iterated, $a_n=H(P_0^{n-1})={\sum }_{k=0}^{n-1}H(T^{-k}P\mid {\bigvee }_{j=0}^{k-1}{T}^{-j}P)$. Measure-preservation makes $H(T^{-k}P\mid {\bigvee }_{j<k}{T}^{-j}P)=H(P\mid {\bigvee }_{j=1}^k{T}^{-j}P)=:c_k$. The sequence $c_k$ is non-increasing in $k$ (conditioning on a finer partition lowers conditional entropy), hence converges to $c_{\infty }=H(P\mid {\bigvee }_{k\ge 1}{T}^{-k}P)$. Cesàro: $a_n/n=\frac{1}{n}{\sum }_{k=0}^{n-1}{c}_k\to c_{\infty }$. So $h(T,P)=c_{\infty }$, the asymptotic uncertainty in the present given the entire past. $\square$

Proposition 4 (entropy is a conjugacy invariant). If $\Phi :(X,\mu ,T)\to (Y,\nu ,S)$ is a measure-isomorphism with $\Phi \circ T=S\circ \Phi$ a.e., then $h_{\mu }(T)=h_{\nu }(S)$.

Proof. $\Phi$ induces a bijection $P\mapsto {\Phi }^{-1}P$ between finite partitions of $Y$ and of $X$ preserving all atom measures, since $\mu ({\Phi }^{-1}E)=\nu (E)$. The intertwining gives ${\Phi }^{-1}(S^{-k}P)=T^{-k}{\Phi }^{-1}P$, so $H_{\mu }\left({\bigvee }_{k<n}{T}^{-k}{\Phi }^{-1}P\right)=H_{\nu }\left({\bigvee }_{k<n}{S}^{-k}P\right)$ for every $n$. Dividing by $n$ and taking limits, $h_{\mu }(T,{\Phi }^{-1}P)=h_{\nu }(S,P)$. Since ${\Phi }^{-1}$ is a bijection on partitions, taking suprema gives $h_{\mu }(T)={\sup }_P{h}_{\mu }(T,{\Phi }^{-1}P)={\sup }_P{h}_{\nu }(S,P)=h_{\nu }(S)$. $\square$

Connections Master

• The measure-preserving-system and recurrence framework 38.04.01 is the substrate on which entropy is built: the Koopman operator, the join of partitions, and the Rokhlin-tower approximation all come from there, and entropy is the quantitative invariant that recurrence and the spectral data could not supply. Where Kac's formula counts return times, Kolmogorov-Sinai entropy counts the exponential growth of distinguishable orbit-names, completing the measure-theoretic portrait of a system.

• The ergodic theorems of Birkhoff, von Neumann, and Kingman 37.02.03 are the analytic engine behind entropy: Fekete subadditivity gives existence of $h(T,P)$, increasing-martingale convergence drives the generator theorem's limit $H(Q\mid P_{-m}^m)\to 0$, and the Shannon-McMillan-Breiman theorem is itself an application of the Birkhoff and Kingman machinery to the information function $-\log \mu (P_0^{n-1}(x))$. Entropy is the subadditive ergodic theorem read as a growth rate of information.

• The $L^p$ and Hilbert-space theory 02.07.06 supplies the convergence framework: the conditional expectations $\mathbb{E}[1_{Q_j}\mid P_{-m}^m]$ converge in $L^1$ by martingale convergence, the information function lies in $L^1$, and the concavity estimates for $\phi (t)=-t\log t$ are Jensen inequalities in the $L^1$ pairing. The completeness that makes these limits exist is the Riesz-Fischer theorem of that unit.

• Topological entropy and the variational principle 38.06.03 take measure-theoretic entropy as one half of the identity $h_{\mathrm{top}}(T)={\sup }_{\mu }{h}_{\mu }(T)$: the topological invariant is the supremum of Kolmogorov-Sinai entropies over invariant measures, and a measure of maximal entropy realises it. The partition-counting of this unit is the measure-side input to that bridge.

• Smooth ergodic theory and the Pesin formula 38.07.01 identify $h_{\mu }(T)=\int {\sum }_i{\lambda }_i^{+}d\mu$ for smooth systems preserving a smooth measure, equating entropy with integrated positive Lyapunov exponents — the toral-automorphism computation $h={\sum }_{|{\lambda }_i|>1}\log |{\lambda }_i|$ of this unit is the linear, constant-exponent prototype of that integral.

Historical & philosophical context Master

The notion of entropy entered dynamics by analogy with Shannon's 1948 information theory ^{[Shannon 1948]}, where $-\sum p_i\log p_i$ measured the average information of a source. The decisive transfer to dynamics was made by Andrei Kolmogorov in two 1958 Doklady notes ^{[Kolmogorov 1958]}, who defined an entropy for measure-preserving automorphisms and recognised it as a conjugacy invariant capable of distinguishing systems that the existing spectral theory could not. Kolmogorov's original definition was technically restricted; Yakov Sinai's 1959 Doklady paper ^{[Sinai 1959]} gave the definition in the form used today — supremum over partitions of the relative entropy — and proved the generator theorem that made the invariant computable.

The immediate triumph was the Bernoulli isomorphism problem. Von Neumann had asked whether the two-shift $B(1/2,1/2)$ and the three-shift $B(1/3,1/3,1/3)$ are isomorphic as measure-preserving systems; both are mixing with countable Lebesgue spectrum, so spectral invariants are powerless. Kolmogorov-Sinai entropy gave $\log 2$ versus $\log 3$, an immediate negative answer. Donald Ornstein's 1970 theorem ^{[Ornstein 1970]} proved the converse — equal entropy implies isomorphism for Bernoulli shifts — establishing entropy as a complete invariant of the Bernoulli class and opening the isomorphism theory that occupied ergodic theory through the 1970s. The connection to smooth dynamics came through the work of Pesin in the 1970s, relating entropy to Lyapunov exponents and tying the abstract invariant to the geometry of hyperbolicity that Anosov and Smale had developed.

Bibliography Master

@article{Kolmogorov1958,
  author  = {Kolmogorov, Andrei N.},
  title   = {A new metric invariant of transient dynamical systems and automorphisms of Lebesgue spaces},
  journal = {Doklady Akademii Nauk SSSR},
  volume  = {119},
  year    = {1958},
  pages   = {861--864}
}

@article{Sinai1959,
  author  = {Sinai, Yakov G.},
  title   = {On the notion of entropy of a dynamical system},
  journal = {Doklady Akademii Nauk SSSR},
  volume  = {124},
  year    = {1959},
  pages   = {768--771}
}

@article{Shannon1948,
  author  = {Shannon, Claude E.},
  title   = {A mathematical theory of communication},
  journal = {Bell System Technical Journal},
  volume  = {27},
  year    = {1948},
  pages   = {379--423, 623--656}
}

@article{Ornstein1970,
  author  = {Ornstein, Donald S.},
  title   = {Bernoulli shifts with the same entropy are isomorphic},
  journal = {Advances in Mathematics},
  volume  = {4},
  number  = {3},
  year    = {1970},
  pages   = {337--352}
}

@book{Walters1982,
  author    = {Walters, Peter},
  title     = {An Introduction to Ergodic Theory},
  publisher = {Springer},
  series    = {Graduate Texts in Mathematics},
  volume    = {79},
  year      = {1982}
}

@book{CornfeldFominSinai1982,
  author    = {Cornfeld, Isaac P. and Fomin, Sergei V. and Sinai, Yakov G.},
  title     = {Ergodic Theory},
  publisher = {Springer},
  series    = {Grundlehren der mathematischen Wissenschaften},
  volume    = {245},
  year      = {1982}
}

@book{Petersen1983,
  author    = {Petersen, Karl},
  title     = {Ergodic Theory},
  publisher = {Cambridge University Press},
  year      = {1983}
}

@book{Glasner2003,
  author    = {Glasner, Eli},
  title     = {Ergodic Theory via Joinings},
  publisher = {American Mathematical Society},
  series    = {Mathematical Surveys and Monographs},
  volume    = {101},
  year      = {2003}
}