The Large Deviation Principle: Rate Functions, Bounds, and Goodness

Intuition Beginner

Flip a fair coin a thousand times and the fraction of heads will sit very near one half. Could it instead land near $0.6$? Yes, but only with a probability so small it is hard to write down. The large deviation principle is the precise statement of how small: not just that rare events are unlikely, but that their unlikeliness shrinks at a clean, predictable exponential rate as the experiment grows.

Picture a whole family of random quantities indexed by a smallness parameter — say the average of $n$ coin flips, with the family getting sharper as $n$ grows. For each possible outcome value, there is a number that measures how costly that value is to reach. Outcomes near the typical answer cost nothing; outcomes far away cost a lot. The probability of landing near a costly value decays like $e^{-n\times (\text{cost})}$. That cost-per-unit-size function is the rate function, and it is the central character of the whole theory.

A useful way to feel it: think of the rate function as the height of a landscape. The typical outcome sits at the bottom of a valley at height zero. To be seen at any other point, the random average must "climb" to that point's height, and the chance of being seen there falls off exponentially in the height times the size of the experiment. Watching where the landscape is low tells you where the random quantity likes to be; watching where it is high tells you which rare events are merely rare versus essentially impossible.

The principle comes in two halves, one for each direction of estimate. The first half says rare events are at most as likely as their cheapest point allows — an upper bound on probability. The second half says they are at least that likely — a matching lower bound. When the two halves agree, the rate function is pinned down exactly, and we say the family obeys a large deviation principle.

One last refinement matters. A rate function is called good when its low-lying regions are not just bounded but compact — the valleys never run off to infinity at a fixed height. Goodness is the technical comfort that makes the optimisation "find the cheapest point in a set" actually attain its minimum, which is what most applications quietly rely on.

Visual Beginner

Figure: a landscape-style plot. The horizontal axis is the value of a random average; the vertical axis is the rate function (the "cost"). The curve dips to height zero at the typical value and rises on both sides like a valley. A horizontal dashed line at height $\alpha$ cuts the valley into a low region (the sublevel set) and two high flanks. For a "good" rate function the low region is a closed bounded interval; an inset shows a "bad" rate function whose valley flattens to a horizontal ray, so its low region runs off to infinity.

  cost I(x)
    |\                                   /
    | \                                 /
    |  \                               /
    |   \                             /
 a -+----\---------------------------/----   sublevel set {I <= a}
    |     \                         /         = the segment between the
    |      \_____               ___/            two crossings (compact
    |            \____     ____/                 when I is "good")
    |                \___ ___/
----+--------------------V------------------- x
    |              typical value (I = 0)

   good rate function: every horizontal cut leaves a compact (closed,
   bounded) low region.  bad rate function: some cut leaves a low region
   that escapes to infinity.

Worked example Beginner

Take the average of $n$ fair coin flips, scoring each flip $1$ for heads and $0$ for tails. The average ${\bar{X}}_n$ is the fraction of heads. We ask how the cost lands for a target fraction $x$ between $0$ and $1$.

Step 1. Name the cost. For fair coins the rate function turns out to be $$ I(x) = x\log(2x) + (1-x)\log\big(2(1-x)\big), $$ the "distance" of a biased coin showing fraction $x$ from a fair coin. We will read values off it; the derivation lives at the higher tiers.

Step 2. Check the typical value. Put $x=\frac{1}{2}$. Then $2x=1$ and $2(1-x)=1$, so both logarithms are $\log 1=0$ and $I(\frac{1}{2})=0$. The typical fraction costs nothing, exactly as the valley-bottom picture demands.

Step 3. Price a rare value. Put $x=0.6$. Then $2x=1.2$ and $2(1-x)=0.8$: $$ I(0.6) = 0.6\log(1.2) + 0.4\log(0.8) = 0.6(0.1823) + 0.4(-0.2231) = 0.1094 - 0.0893 = 0.0201. $$

Step 4. Read off the probability scale. With $n=1000$ flips, the chance of seeing roughly $60\%$ heads decays like $$ e^{-n,I(0.6)} = e^{-1000 \times 0.0201} = e^{-20.1} \approx 1.9 \times 10^{-9}. $$

What this tells us. A modest-looking shift, from $50\%$ to $60\%$ heads, already costs about two parts in a billion at a thousand flips — and at ten thousand flips it would be $e^{-201}$, astronomically smaller. The rate function converts a vague "very unlikely" into an exact exponential rate, and that conversion is the entire point of the large deviation principle.

Check your understanding Beginner

Formal definition Intermediate+

Throughout, $\mathcal{X}$ is a topological space equipped with its Borel $\sigma$-algebra ${\mathcal{B}}_{\mathcal{X}}$ (regular and Hausdorff where uniqueness is asserted), and $\{{\mu }_{\epsilon }{\}}_{\epsilon >0}$ is a family of Borel probability measures on $\mathcal{X}$, carried on a common probability space 37.01.01. A speed is a family $\{a_{\epsilon }{\}}_{\epsilon >0}$ of positive reals with $a_{\epsilon }\to 0$ as $\epsilon \to 0$ (in the index-by-$n$ convention one takes $\epsilon =1/n$ and $a_{\epsilon }=1/n$, so $a_{\epsilon }\log {\mu }_{\epsilon }\approx \frac{1}{n}\log \mathbb{P}$). The role of $a_{\epsilon }$ is to be the reciprocal of the size of the experiment.

Definition (rate function). A rate function is a lower-semicontinuous map $I:\mathcal{X}\to [0,+\infty ]$, not identically $+\infty$; equivalently, every sublevel set $$ \Psi_I(\alpha) := {x \in \mathcal{X} : I(x) \leq \alpha}, \qquad \alpha \geq 0, $$ is closed. A rate function is good if in addition every ${\Psi }_I(\alpha )$ is compact. The effective domain is ${\mathcal{D}}_I:=\{x:I(x)<\infty \}$.

Definition (large deviation principle). The family $\{{\mu }_{\epsilon }\}$ satisfies a large deviation principle (LDP) at speed $a_{\epsilon }$ with rate function $I$ if, for every Borel set $\Gamma$, writing ${\Gamma }^{\circ }$ for its interior and $\overline{\Gamma }$ for its closure, $$ -\inf_{x \in \Gamma^\circ} I(x) ;;\leq;; \liminf_{\varepsilon \to 0} a_\varepsilon \log \mu_\varepsilon(\Gamma) ;;\leq;; \limsup_{\varepsilon \to 0} a_\varepsilon \log \mu_\varepsilon(\Gamma) ;;\leq;; -\inf_{x \in \overline{\Gamma}} I(x). $$ Unpacked, the LDP is the conjunction of two bounds ^{[Dembo & Zeitouni §1.2]}:

• (Lower bound — open sets.) For every open $G\subseteq \mathcal{X}$,

  $$\underset{\epsilon \to 0}{\mathrm{liminf}}{a}_{\epsilon }\log {\mu }_{\epsilon }(G)\ge -\underset{x\in G}{\inf }I(x).$$

• (Upper bound — closed sets.) For every closed $F\subseteq \mathcal{X}$,

  $$\underset{\epsilon \to 0}{\mathrm{limsup}}{a}_{\epsilon }\log {\mu }_{\epsilon }(F)\le -\underset{x\in F}{\inf }I(x).$$

The convention ${\inf }_{\varnothing }I=+\infty$ makes both bounds vacuously hold on the empty set. For a set $\Gamma$ that is an $I$-continuity set, meaning ${\inf }_{{\Gamma }^{\circ }}I={\inf }_{\overline{\Gamma }}I=:I(\Gamma )$, the two bounds collapse to the exact exponential rate ${\lim }_{\epsilon }{a}_{\epsilon }\log {\mu }_{\epsilon }(\Gamma )=-I(\Gamma )$.

Definition (weak LDP). The family satisfies a weak LDP if the lower bound holds for all open sets and the upper bound holds only for all compact sets $K$: $$ \limsup_{\varepsilon \to 0} a_\varepsilon \log \mu_\varepsilon(K) ;\leq; -\inf_{x \in K} I(x), \qquad K \text{ compact}. $$ The weak LDP is genuinely weaker: it controls probabilities of escape only into compact regions, saying nothing about mass that leaks to infinity. The device that closes this gap is the following.

Definition (exponential tightness). The family $\{{\mu }_{\epsilon }\}$ is exponentially tight if for every $M<\infty$ there is a compact $K_M\subseteq \mathcal{X}$ with $$ \limsup_{\varepsilon \to 0} a_\varepsilon \log \mu_\varepsilon\big(K_M^{,c}\big) ;\leq; -M. $$ Exponential tightness is the large-deviation analogue of ordinary tightness: ordinary tightness controls the mass outside compacts, exponential tightness controls its exponential rate. It is the bridge that upgrades a weak LDP to a full LDP, and it forces the rate function to be good.

Counterexamples to common slips

• The open/closed asymmetry is not cosmetic. Take $\mathcal{X}=\mathbb{R}$ and the deterministic family ${\mu }_{\epsilon }={\delta }_0$ for all $\epsilon$, with $I(0)=0$, $I(x)=+\infty$ otherwise. The closed set $F=\{0\}$ has ${\mu }_{\epsilon }(F)=1$ and $-{\inf }_{F}I=0$: the upper bound is tight. But for the open set $G=(0,1)$, ${\mu }_{\epsilon }(G)=0$ and $a_{\epsilon }\log 0=-\infty =-{\inf }_{G}I$. Evaluate the upper bound on the open interval's closure $\overline{G}=[0,1]$ instead and you would wrongly get $0$. The two bounds must be read on the correct topological side.
• Lower semicontinuity is mandatory; a non-lsc "rate function" is not unique. If one drops lower semicontinuity, the value of $I$ at a single point can be lowered to $0$ on a set of $I$-measure-zero topological negligibility without changing any ${\inf }_G$ over open $G$, so the bounds cannot pin $I$ down pointwise. Lower semicontinuity is exactly the regularity that the infima over shrinking open neighbourhoods recover, which is why uniqueness holds only within the class of lsc functions.
• Good is strictly stronger than lsc on non-compact spaces. On $\mathcal{X}=\mathbb{R}$ the function $I(x)=0$ for all $x$ is lsc but not good: ${\Psi }_I(0)=\mathbb{R}$ is closed but not compact. A rate function can be a perfectly valid lsc rate function while failing goodness, and then the variational problem ${\inf }_{F}I$ over a closed unbounded $F$ may not be attained.

Key theorem with proof Intermediate+

We prove the two structural pillars an LDP user relies on first: that the rate function is determined by the family (uniqueness), and that exponential tightness plus a weak LDP delivers the full LDP and goodness.

Theorem (uniqueness and the weak-to-full upgrade). Let $\{{\mu }_{\epsilon }\}$ be Borel probability measures on a Hausdorff regular space $\mathcal{X}$ at speed $a_{\epsilon }$.

(i) (Uniqueness.) If $\{{\mu }_{\epsilon }\}$ satisfies the LDP with rate function $I$ and also with rate function $J$, and both are lower-semicontinuous, then $I=J$.

(ii) (Upgrade.) If $\{{\mu }_{\epsilon }\}$ satisfies a weak LDP with rate function $I$ and is exponentially tight, then it satisfies the full LDP with the same $I$, and $I$ is good.

Proof of (i). Fix $x\in \mathcal{X}$. For any open neighbourhood $G\ni x$, the lower bound with rate $I$ and the upper bound with rate $J$ (applied to $\overline{G}$) give $$ -\inf_{G} I ;\leq; \liminf_\varepsilon a_\varepsilon\log\mu_\varepsilon(G) ;\leq; \limsup_\varepsilon a_\varepsilon\log\mu_\varepsilon(\overline G) ;\leq; -\inf_{\overline G} J ;\leq; -\inf_{G} J, $$ the last step because $G\subseteq \overline{G}$ makes the infimum over the larger set no larger. Hence ${\inf }_{G}J\le {\inf }_{G}I\le I(x)$ for every open $G\ni x$. Taking the supremum over neighbourhoods and using lower semicontinuity of $J$, which states $J(x)={\sup }_{G\ni x}{\inf }_{G}J$, gives $J(x)\le I(x)$. The roles of $I$ and $J$ are symmetric, so $I(x)\le J(x)$ as well, and $I=J$. $\square$

Proof of (ii). The lower bound is already part of the weak LDP, so only the upper bound on a general closed set $F$ needs proof, and goodness needs to be derived. Fix $M<\infty$ and choose the compact $K_M$ from exponential tightness. Decompose $F=(F\cap K_M)\cup (F\cap K_M^c)$. The set $F\cap K_M$ is compact (closed subset of a compact set), so the weak LDP applies to it: $$ \limsup_\varepsilon a_\varepsilon\log\mu_\varepsilon(F\cap K_M) ;\leq; -\inf_{F\cap K_M} I ;\leq; -\inf_{F} I. $$ For the tail piece, ${\mu }_{\epsilon }(F\cap K_M^c)\le {\mu }_{\epsilon }(K_M^c)$, so ${\mathrm{limsup}}_{\epsilon }{a}_{\epsilon }\log {\mu }_{\epsilon }(F\cap K_M^c)\le -M$. Combining two exponential rates uses the elementary bound $a_{\epsilon }\log (p+q)\le a_{\epsilon }\log 2+\max \{a_{\epsilon }\log p,a_{\epsilon }\log q\}$, and $a_{\epsilon }\log 2\to 0$; therefore $$ \limsup_\varepsilon a_\varepsilon\log\mu_\varepsilon(F) ;\leq; \max\Big{ -\inf_{F} I,; -M\Big}. $$ Letting $M\to \infty$ yields ${\mathrm{limsup}}_{\epsilon }{a}_{\epsilon }\log {\mu }_{\epsilon }(F)\le -{\inf }_{F}I$, the full upper bound.

For goodness, fix $\alpha \ge 0$ and take $M>\alpha$ with its compact $K_M$. If $x\notin K_M$, then $\{x\}$ has an open neighbourhood $G$ disjoint from $K_M$ (regularity), and the upper bound just proved on the closed set $\overline{G}\subseteq K_M^c$ gives $-{\inf }_{\overline{G}}I\le -M<-\alpha$, so ${\inf }_{\overline{G}}I>\alpha$ and in particular $I(x)>\alpha$. Contraposing, ${\Psi }_I(\alpha )=\{I\le \alpha \}\subseteq K_M$. Being a closed subset (lsc) of the compact $K_M$, the sublevel set ${\Psi }_I(\alpha )$ is compact. Hence $I$ is good. $\square$

Bridge. This theorem builds toward every concrete large-deviation result and appears again in the proof of Cramér's theorem, where one first establishes the weak LDP from the Chernoff bound and a local lower bound, then invokes exponential tightness from the finiteness of the cumulant generating function near the origin to obtain the full principle. This is exactly the mechanism by which the abstract axioms become usable: the weak LDP is what the Chernoff/tilting estimates naturally produce, and exponential tightness is the separate, geometric input that compactifies the problem. The foundational reason uniqueness holds is that lower semicontinuity makes a rate function the upper envelope of its values over shrinking open neighbourhoods, so the open-set lower bound and the closed-set upper bound between them recover $I$ pointwise; putting these together with the upgrade shows that exponential tightness is precisely the hypothesis that generalises the compact-set control of the weak LDP to all closed sets and simultaneously forces goodness. The weak-LDP-plus-tightness route is dual to the projective-limit (Dawson-Gärtner) route, which builds the same full LDP from finite-dimensional marginals instead of from a tightness estimate.

Exercises Intermediate+

Advanced results Master

The rate function as a generating object: Varadhan's lemma

Once an LDP with good rate $I$ holds for $\{{\mu }_{\epsilon }\}$ at speed $a_{\epsilon }$, the rate function controls not only probabilities but exponential integrals. Varadhan's lemma ^{[Varadhan 1984]} states that for a continuous $\varphi :\mathcal{X}\to \mathbb{R}$ satisfying the moment condition ${\mathrm{limsup}}_{\epsilon }{a}_{\epsilon }\log \int e^{\gamma \varphi /a_{\epsilon }}d{\mu }_{\epsilon }<\infty$ for some $\gamma >1$, $$ \lim_{\varepsilon\to0} a_\varepsilon \log \int_{\mathcal{X}} e^{\phi(x)/a_\varepsilon},\mu_\varepsilon(dx) ;=; \sup_{x\in\mathcal{X}}\big(\phi(x) - I(x)\big). $$ This is the infinite-dimensional, probabilistic Laplace method: the integral is dominated by the point where the gain $\varphi$ most exceeds the cost $I$, a Legendre-Fenchel-type pairing of the functional $\varphi$ against the rate function. The good-rate-function hypothesis is what makes the supremum attained and the upper estimate uniform over level sets of $I$.

Inverse Varadhan and Bryc's lemma

The implication reverses under exponential tightness. Bryc's lemma states that if $\{{\mu }_{\epsilon }\}$ is exponentially tight and the limit $\Lambda (\varphi ):={\lim }_{\epsilon }{a}_{\epsilon }\log \int e^{\varphi /a_{\epsilon }}d{\mu }_{\epsilon }$ exists for every bounded continuous $\varphi$, then $\{{\mu }_{\epsilon }\}$ satisfies the LDP with good rate function $$ I(x) = \sup_{\phi\in C_b(\mathcal{X})}\big(\phi(x) - \Lambda(\phi)\big), $$ the Legendre-Fenchel transform of $\Lambda$ over the space of bounded continuous test functions. Thus the LDP and the convergence of all exponential moments are equivalent data, modulo exponential tightness, and the rate function is recovered as a conjugate — the abstract shadow of the cumulant-conjugate identity of the Cramér theory.

Uniqueness sharpened: the role of the topology

Uniqueness of the rate function (Key theorem (i)) is a statement about a fixed topology on $\mathcal{X}$. Strengthening the topology enlarges the class of open sets, hence tightens the lower bound, and can increase the rate function pointwise; weakening it can decrease it. A family may satisfy an LDP in the weak topology with rate $I_w$ and in the strong topology with rate $I_s\ge I_w$, the two agreeing on a common core but differing at the boundary of the effective domain. This is the precise sense in which "the" rate function is topology-relative, and it is why Sanov's theorem distinguishes the weak and $\tau$-topologies on the space of empirical measures.

Exponential tightness from exponential moment bounds

In practice exponential tightness is verified through a coercive functional. If there is a function $U:\mathcal{X}\to [0,\infty ]$ with compact sublevel sets and ${\mathrm{limsup}}_{\epsilon }{a}_{\epsilon }\log \int e^{U/a_{\epsilon }}d{\mu }_{\epsilon }<\infty$, then $\{{\mu }_{\epsilon }\}$ is exponentially tight: the set $K_M=\{U\le c_M\}$ is compact, and Markov's inequality on $e^{U/a_{\epsilon }}$ bounds ${\mu }_{\epsilon }(K_M^c)$ at the required exponential rate. For Cramér's theorem on ${\mathbb{R}}^d$, $U(x)=\Vert x\Vert$ works whenever $\Lambda$ is finite in a neighbourhood of $0$; for Schilder's theorem on path space the coercive functional is built from the Cameron-Martin norm, and exponential tightness is the Arzelà-Ascoli equicontinuity estimate transported to the exponential scale.

Schilder's theorem: a preview on path space

The canonical infinite-dimensional instance is Schilder's theorem ^{[Schilder 1966]}. For $\epsilon$-scaled Brownian motion $\sqrt{\epsilon }W$ on $C_0([0,1])$, the laws ${\mu }_{\epsilon }=\mathrm{Law}(\sqrt{\epsilon }W)$ satisfy an LDP at speed $\epsilon$ with good rate function $$ I(f) = \tfrac12\int_0^1 |\dot f(t)|^2,dt $$ on the Cameron-Martin space of absolutely continuous $f$ with $f(0)=0$ and square-integrable derivative, and $I(f)=+\infty$ otherwise. The cost of a Brownian path of small noise following a prescribed shape $f$ is its kinetic-energy action — the same Onsager-Machlup action that reappears in the path-integral treatment of fluctuations. Goodness of $I$ here is the statement that bounded-action paths form a compact set in the uniform topology, an Arzelà-Ascoli fact dressed exponentially.

Synthesis. The central insight of this unit is that a single lower-semicontinuous cost function $I$ governs a whole family of exponential asymptotics, and the LDP axioms are the minimal bookkeeping that generalises the elementary Cramér exponent to arbitrary topological spaces. This is exactly why the weak LDP and exponential tightness are separated: the weak LDP is the local, tilting-driven content that appears again in every change-of-measure lower bound and Chernoff upper bound, while exponential tightness is the global compactness input that upgrades it and is dual to ordinary tightness in the theory of weak convergence. The foundational reason the rate function is unique and good is the interplay of lower semicontinuity with the open/closed asymmetry of the two bounds: lsc makes $I$ the upper envelope of neighbourhood infima, and exponential tightness pins its sublevel sets inside fixed compacta. Putting these together, Varadhan's lemma and its Bryc inverse show that the LDP, the rate function, and the convergence of all exponential moments are three encodings of one datum, with the rate function recovered as a Legendre-Fenchel conjugate — the bridge is the conjugacy that the next unit 37.07.03 makes explicit, where $I={\Lambda }^{\ast }$ for the Cramér cumulant generating function $\Lambda$.

Full proof set Master

Proposition 1 (lower semicontinuity is forced by the lower bound). Suppose the open-set lower bound holds for some function $I:\mathcal{X}\to [0,\infty ]$. Then it also holds for the lower-semicontinuous regularisation $I_{\mathrm{lsc}}(x):={\sup }_{G\ni x\text{ open}}{\inf }_{y\in G}I(y)\le I(x)$, and $I_{\mathrm{lsc}}$ is lsc. Hence one may always take the rate function lower-semicontinuous.

Proof. From the definition, $I_{\mathrm{lsc}}(x)={\sup }_{G\ni x}{\inf }_{G}I\le I(x)$, since one admissible neighbourhood is any $G$ containing $x$ and each ${\inf }_{G}I\le I(x)$. Because $I_{\mathrm{lsc}}\le I$ pointwise, ${\inf }_G{I}_{\mathrm{lsc}}\le {\inf }_{G}I$ for every open $G$, so $-{\inf }_G{I}_{\mathrm{lsc}}\ge -{\inf }_{G}I$ and the lower bound $\mathrm{liminf}{a}_{\epsilon }\log {\mu }_{\epsilon }(G)\ge -{\inf }_{G}I\ge -{\inf }_G{I}_{\mathrm{lsc}}$ holds with $I_{\mathrm{lsc}}$ in place of $I$. Lower semicontinuity of $I_{\mathrm{lsc}}$: for $\alpha \in \mathbb{R}$ and $x$ with $I_{\mathrm{lsc}}(x)>\alpha$, by definition some open $G\ni x$ has ${\inf }_{G}I>\alpha$, whence $I_{\mathrm{lsc}}(z)\ge {\inf }_{G}I>\alpha$ for all $z\in G$; so $\{I_{\mathrm{lsc}}>\alpha \}$ is open and $\{I_{\mathrm{lsc}}\le \alpha \}$ closed. $\square$

Proposition 2 (the upper bound on compacts gives the upper bound on closed $I$-bounded sets under exponential tightness). Let the weak LDP hold with rate $I$ and let $\{{\mu }_{\epsilon }\}$ be exponentially tight. Then for every closed $F$, ${\mathrm{limsup}}_{\epsilon }{a}_{\epsilon }\log {\mu }_{\epsilon }(F)\le -{\inf }_{F}I$.

Proof. This is the upgrade clause of the Key theorem; we record it as a standalone proposition with the combination step made explicit. Fix $M$ and the compact $K_M$. Then $F=(F\cap K_M)\cup (F\cap K_M^c)$ and by finite subadditivity ${\mu }_{\epsilon }(F)\le {\mu }_{\epsilon }(F\cap K_M)+{\mu }_{\epsilon }(K_M^c)$. For any two non-negative sequences $p_{\epsilon },q_{\epsilon }$, $a_{\epsilon }\log (p_{\epsilon }+q_{\epsilon })\le a_{\epsilon }\log 2+\max \{a_{\epsilon }\log p_{\epsilon },a_{\epsilon }\log q_{\epsilon }\}$, and $a_{\epsilon }\log 2\to 0$. Applying with $p_{\epsilon }={\mu }_{\epsilon }(F\cap K_M)$, $q_{\epsilon }={\mu }_{\epsilon }(K_M^c)$ and the compact upper bound $\mathrm{limsup}{a}_{\epsilon }\log {\mu }_{\epsilon }(F\cap K_M)\le -{\inf }_{F\cap K_M}I\le -{\inf }_{F}I$ together with $\mathrm{limsup}{a}_{\epsilon }\log {\mu }_{\epsilon }(K_M^c)\le -M$, $$ \limsup_\varepsilon a_\varepsilon\log\mu_\varepsilon(F) \le \max{-\inf_F I, -M}. $$ Take $M\uparrow \infty$. $\square$

Proposition 3 (the rate function vanishes somewhere when ${\mu }_{\epsilon }$ are probability measures). If $\{{\mu }_{\epsilon }\}$ are probability measures satisfying the LDP with rate $I$, then ${\inf }_{x\in \mathcal{X}}I(x)=0$.

Proof. Apply the upper bound to the closed set $F=\mathcal{X}$: ${\mathrm{limsup}}_{\epsilon }{a}_{\epsilon }\log {\mu }_{\epsilon }(\mathcal{X})\le -{\inf }_{\mathcal{X}}I$. The left side is $a_{\epsilon }\log 1=0$, so $-{\inf }_{\mathcal{X}}I\ge 0$, i.e. ${\inf }_{\mathcal{X}}I\le 0$; since $I\ge 0$, ${\inf }_{\mathcal{X}}I=0$. If moreover $I$ is good, the infimum is attained at some $x_{\ast }$ with $I(x_{\ast })=0$, the large-deviation expression of the law of large numbers: the family concentrates exponentially on the zero set of $I$. $\square$

Connections Master

• The rate function defined here is identified concretely as a Legendre-Fenchel conjugate in 37.07.03: for the Cramér family the rate function is ${\Lambda }^{\ast }$, the convex conjugate of the cumulant generating function $\Lambda (\lambda )=\log \mathbb{E}{e}^{⟨\lambda ,X⟩}$, and the goodness this unit demands abstractly is the compact-sublevel-set property that the convex-duality unit proves from finiteness of $\Lambda$ near the origin.

• The exponential-tightness-plus-weak-LDP machinery is the engine of the Gärtner-Ellis theorem 37.07.04: that theorem produces a weak LDP from the differentiability and steepness of the limiting cumulant generating function, then closes to a full LDP exactly via the exponential tightness isolated here, so the bridge concept of this unit is the load-bearing hypothesis there.

• The change-of-measure lower bound that powers every Cramér-type LDP is an exponential tilt $d{\mu }_{\epsilon }^{\lambda }/d{\mu }_{\epsilon }\propto e^{⟨\lambda ,\cdot ⟩/a_{\epsilon }}$, a Radon-Nikodym derivative in the sense of 02.07.08; absolute continuity of the tilted family with respect to the original is what licenses transferring probability estimates across the tilt.

• The liminf/limsup bookkeeping of the two LDP bounds, and the passage of the lower bound through integrals in Varadhan's lemma, rest on the dominated-convergence and Fatou control of 02.07.05; the carrier probability space on which the whole family $\{{\mu }_{\epsilon }\}$ is realised is the Kolmogorov construction of 37.01.01.

Historical & philosophical context Master

The exponential decay of rare-event probabilities was first computed by Harald Cramér in 1938 ^{[Cramér 1938]}, who found the rate for sums of i.i.d. variables under an analytic-density assumption and identified it as the conjugate of the cumulant generating function. Independent threads ran through Khinchin and the early statistical-mechanics literature on entropy, where the same exponential-of-an-extensive-quantity structure appears as $W\approx e^{S/k_B}$. The abstract formulation — an LDP as a pair of bounds indexed by open and closed sets, governed by a single lower-semicontinuous rate function — is due to S. R. S. Varadhan in 1966 ^{[Varadhan 1966]}, who in the same period proved the integral lemma now bearing his name; this is the formulation systematised in Dembo and Zeitouni ^{[Dembo & Zeitouni §1.2]}.

The path-space instance was settled the same year by Schilder ^{[Schilder 1966]}, whose small-noise asymptotics for Wiener integrals gave the kinetic-action rate function and seeded the Freidlin-Wentzell theory of randomly perturbed dynamical systems. The notions of weak LDP and exponential tightness were isolated as the right abstraction by Deuschel and Stroock and by Dembo and Zeitouni, reflecting the recognition that the local (tilting) content of a large-deviation estimate and the global (compactness) content are logically separate, the second supplied by tightness exactly as in the Prokhorov theory of weak convergence. The goodness condition formalises the requirement, implicit since Cramér, that the variational problems ${\inf }_{F}I$ attached to an LDP actually attain their minima.

Bibliography Master

@book{dembozeitouni1998ldp,
  author    = {Dembo, Amir and Zeitouni, Ofer},
  title     = {Large Deviations Techniques and Applications},
  edition   = {2nd},
  series    = {Applications of Mathematics},
  number    = {38},
  publisher = {Springer},
  year      = {1998}
}

@book{varadhan1984large,
  author    = {Varadhan, S. R. S.},
  title     = {Large Deviations and Applications},
  series    = {CBMS-NSF Regional Conference Series in Applied Mathematics},
  number    = {46},
  publisher = {SIAM},
  year      = {1984}
}

@article{varadhan1966asymptotic,
  author  = {Varadhan, S. R. S.},
  title   = {Asymptotic probabilities and differential equations},
  journal = {Communications on Pure and Applied Mathematics},
  volume  = {19},
  pages   = {261--286},
  year    = {1966}
}

@article{schilder1966asymptotic,
  author  = {Schilder, M.},
  title   = {Some asymptotic formulas for {W}iener integrals},
  journal = {Transactions of the American Mathematical Society},
  volume  = {125},
  pages   = {63--85},
  year    = {1966}
}

@article{cramer1938nouveau,
  author  = {Cram\'er, Harald},
  title   = {Sur un nouveau th\'eor\`eme-limite de la th\'eorie des probabilit\'es},
  journal = {Actualit\'es Scientifiques et Industrielles},
  volume  = {736},
  pages   = {5--23},
  year    = {1938}
}

@book{deuschelstroock1989large,
  author    = {Deuschel, Jean-Dominique and Stroock, Daniel W.},
  title     = {Large Deviations},
  series    = {Pure and Applied Mathematics},
  number    = {137},
  publisher = {Academic Press},
  year      = {1989}
}

@book{denhollander2000large,
  author    = {den Hollander, Frank},
  title     = {Large Deviations},
  series    = {Fields Institute Monographs},
  number    = {14},
  publisher = {American Mathematical Society},
  year      = {2000}
}