The Legendre-Fenchel Transform and Convex Duality of Rate Functions

Intuition Beginner

Imagine a function $f$ as a hill: at each location $x$ it reports a height $f(x)$. Most of the time you describe the hill by listing heights position by position. But there is a second, equally complete way to describe it: record, for every possible slope $\lambda$, the tangent line of that slope that touches the hill from below, and write down how far that line dips below zero. This second description, indexed by slope instead of position, is the conjugate function $f^{\ast }$.

Why bother with a slope-based description? Because some questions are easier in slope language. In probability, the rare-event behaviour of an average is controlled by a quantity called the cumulant generating function, which is naturally a function of a "tilting" slope $\lambda$. The cost of seeing a rare value — the rate function — is its conjugate. Switching between the two is the whole game.

You have met a version of this before. In mechanics, the Legendre transform swaps velocity for momentum; in thermodynamics, it swaps entropy for temperature. Those used smooth, bowl-shaped functions where each slope occurs exactly once, so the swap is reversible and clean. The Legendre-Fenchel transform is the rugged-terrain version. It works even when the hill has flat stretches, sharp corners, or vertical cliffs, where a single slope might touch a whole flat segment, or where no tangent line of a given slope exists at all.

The defining recipe is a maximisation:

$$f^{\ast }(\lambda )=\underset{x}{\max }(\lambda x-f(x)).$$

For each slope $\lambda$, you tilt the hill by subtracting the straight line $\lambda x$, then ask how high the tilted hill rises. That highest value is the conjugate. No derivatives, no inverting — just a maximisation, which always makes sense even on rough terrain.

The miracle is that for nicely shaped (convex, downward-cupping-free) hills, doing this twice returns you exactly where you started: $f^{\ast \ast }=f$. The slope-description loses nothing. This round-trip is the source of "convex duality," and in probability it is the reason the rate function and the cumulant generating function are two faces of one object.

Visual Beginner

Figure: a convex curve $f(x)$ with several tangent lines drawn beneath it. Each tangent line has a slope $\lambda$ and crosses the vertical axis at some height $-f^(\lambda)$. As the slope is dialed up, the tangent rolls along the underside of the curve, and the family of all these supporting lines reconstructs the curve completely. A second small panel shows a curve with a flat segment: there, one single slope supports the whole flat stretch, and the conjugate develops a corner.*

   f(x)                          conjugate viewpoint
    |        .                    every slope lambda picks
    |       .                     the tangent line that
    |      .                      touches f from below;
    |    ..                       its axis-crossing is -f*(lambda)
    |  ..    \  tangent slope L
    |..       \                   slope L  ->  intercept -f*(L)
----+-----------\----- x         ------------------------------
    |            \                a flat piece of f  ->  a corner of f*
    |  (intercept = -f*(L))       a corner of f      ->  a flat piece of f*

Worked example Beginner

Take the simplest interesting hill, a parabola $f(x)=\frac{1}{2}{x}^2$. We compute its conjugate by the recipe and check the round-trip.

Step 1. Write down the tilted hill. For a fixed slope $\lambda$, form $g(x)=\lambda x-f(x)=\lambda x-\frac{1}{2}{x}^2$. We want the largest value of $g$ over all $x$.

Step 2. Find the top of the tilted hill. The tilted hill $g(x)=\lambda x-\frac{1}{2}{x}^2$ is itself a downward parabola. Its peak sits where the slope of $g$ is zero. The slope of $g$ is $\lambda -x$, which is zero at $x=\lambda$.

Step 3. Read off the height. Plug $x=\lambda$ back in:

$$f^{\ast }(\lambda )=\lambda \cdot \lambda -\frac{1}{2}{\lambda }^2=\frac{1}{2}{\lambda }^2.$$

So the conjugate of $\frac{1}{2}{x}^2$ is $\frac{1}{2}{\lambda }^2$ — the same shape, now read in slope language. The parabola is its own conjugate.

Step 4. Round-trip check. Conjugating again, $f^{\ast \ast }(x)={\max }_{\lambda }(\lambda x-\frac{1}{2}{\lambda }^2)$, which by the identical computation gives $\frac{1}{2}{x}^2=f(x)$. We returned to the start.

What this tells us. The conjugate is genuinely a faithful re-encoding: nothing was lost, because the round-trip recovered $f$ exactly. For the parabola the two descriptions even look identical, but that is special. For a lopsided hill the conjugate has a different shape, yet the round-trip still lands back on $f$ whenever $f$ is convex.

Check your understanding Beginner

Formal definition Intermediate+

Let $f:{\mathbb{R}}^d\to (-\infty ,+\infty ]$ be a function, not identically $+\infty$, with effective domain $\mathrm{dom}f=\{x:f(x)<+\infty \}$. Such an $f$ is proper if $\mathrm{dom}f\ne \varnothing$ and $f$ never takes the value $-\infty$. It is lower-semicontinuous (lsc) if $\{x:f(x)\le \alpha \}$ is closed for every $\alpha \in \mathbb{R}$, equivalently if $f(x)\le {\mathrm{liminf}}_{y\to x}f(y)$ everywhere. It is convex if its epigraph $\mathrm{epi}f=\{(x,t):t\ge f(x)\}$ is a convex subset of ${\mathbb{R}}^{d+1}$.

Definition (Fenchel conjugate). The Legendre-Fenchel transform (or convex conjugate) of $f$ is the function $f^{\ast }:{\mathbb{R}}^d\to (-\infty ,+\infty ]$ defined by

$$\boxed{f^{\ast }(\lambda ):=\underset{x\in {\mathbb{R}}^d}{\sup }(⟨\lambda ,x⟩-f(x)),\lambda \in {\mathbb{R}}^d.}$$

Here $⟨\lambda ,x⟩={\sum }_{i=1}^d{\lambda }_i{x}_i$ is the standard pairing. The supremum is over $x\in \mathrm{dom}f$; it may equal $+\infty$ for some $\lambda$. As a pointwise supremum of the affine functions $\lambda \mapsto ⟨\lambda ,x⟩-f(x)$, the conjugate $f^{\ast }$ is always convex and lsc, regardless of any regularity of $f$ ^{[Rockafellar §12]}.

The biconjugate $f^{\ast \ast }:=(f^{\ast }{)}^{\ast }$ is obtained by conjugating once more:

$$f^{\ast \ast }(x)=\underset{\lambda \in {\mathbb{R}}^d}{\sup }(⟨\lambda ,x⟩-f^{\ast }(\lambda )).$$

A convex function is closed if it is lsc; equivalently its epigraph is a closed convex set. (The only proper convex function that is lsc but whose epigraph would need separate handling is the constant $+\infty$, excluded by properness.)

Definition (essential smoothness and steepness). A proper convex $f$ is essentially smooth if the interior $\mathrm{int}(\mathrm{dom}f)$ is non-empty, $f$ is differentiable there, and $f$ is steep: $|\nabla f(x_k)|\to \infty$ whenever $x_k\to x_0\in \partial (\mathrm{dom}f)$. It is essentially strictly convex if it is strictly convex on every convex subset of $\{x:\partial f(x)\ne \varnothing \}$. A closed proper convex $f$ is a Legendre function if it is both essentially smooth and essentially strictly convex; for such $f$ the gradient $\nabla f$ is a bijection of $\mathrm{int}(\mathrm{dom}f)$ onto $\mathrm{int}(\mathrm{dom}{f}^{\ast })$, with inverse $\nabla f^{\ast }$, and the classical Legendre transform of 09.04.01 and 11.01.02 is recovered exactly ^{[Rockafellar §26]}.

Counterexamples to common slips

• The conjugate of a non-convex $f$ is the conjugate of its convex envelope. If $f(x)=\min (x^2,1)$ (non-convex), then $f^{\ast }$ coincides with the conjugate of the closed convex hull $\mathrm{conv}f$, and $f^{\ast \ast }=\mathrm{conv}f\ne f$. Conjugation cannot detect the non-convex dimple; it always returns a convex function.
• A corner of $f$ becomes a flat piece of $f^{\ast }$, and vice versa. For $f(x)=|x|$, the conjugate is the indicator $f^{\ast }(\lambda )=0$ for $|\lambda |\le 1$ and $+\infty$ otherwise — a flat plateau with a vertical wall. The non-differentiable corner of $|x|$ at $0$ corresponds to the flat segment $[-1,1]$ in the domain of $f^{\ast }$. Differentiability of one side is dual to strict convexity of the other.
• Steepness is not optional in the bijection. The function $f(x)=\sqrt{1+x^2}$ is smooth and strictly convex but not steep: $\nabla f(x)=x/\sqrt{1+x^2}\in (-1,1)$ stays bounded. Its conjugate $f^{\ast }(\lambda )=-\sqrt{1-{\lambda }^2}$ has effective domain only $[-1,1]$, so $\nabla f$ is not onto $\mathbb{R}$. Steepness is exactly what forces $\mathrm{dom}{f}^{\ast }={\mathbb{R}}^d$.

Key theorem with proof Intermediate+

We prove the two structural pillars: the Fenchel-Young inequality, and the Fenchel-Moreau biconjugation theorem, the latter resting on the geometric Hahn-Banach separation of 02.11.02.

Theorem (Fenchel-Young inequality and biconjugation). Let $f:{\mathbb{R}}^d\to (-\infty ,+\infty ]$ be proper.

(i) (Fenchel-Young.) For all $x,\lambda \in {\mathbb{R}}^d$,

$$f(x)+f^{\ast }(\lambda )\ge ⟨\lambda ,x⟩,$$

with equality if and only if $\lambda \in \partial f(x)$, the subdifferential of $f$ at $x$.

(ii) (Fenchel-Moreau.) If in addition $f$ is convex and lsc, then $f^{\ast \ast }=f$.

Proof of (i). By the definition of the supremum, for every $x$,

$$f^{\ast }(\lambda )=\underset{y}{\sup }(⟨\lambda ,y⟩-f(y))\ge ⟨\lambda ,x⟩-f(x),$$

which rearranges to $f(x)+f^{\ast }(\lambda )\ge ⟨\lambda ,x⟩$. Equality holds precisely when $x$ attains the supremum, i.e. when $⟨\lambda ,y⟩-f(y)$ is maximised at $y=x$. That maximisation condition is $f(y)\ge f(x)+⟨\lambda ,y-x⟩$ for all $y$, which is exactly the statement $\lambda \in \partial f(x)$. $\square$

Proof of (ii). The inequality $f^{\ast \ast }\le f$ is immediate from (i): for every $x$,

$$f^{\ast \ast }(x)=\underset{\lambda }{\sup }(⟨\lambda ,x⟩-f^{\ast }(\lambda ))\le \underset{\lambda }{\sup }(⟨\lambda ,x⟩-(⟨\lambda ,x⟩-f(x)))=f(x),$$

since $f^{\ast }(\lambda )\ge ⟨\lambda ,x⟩-f(x)$ for each $\lambda$. The content is the reverse inequality $f^{\ast \ast }\ge f$.

Suppose, for contradiction, that $f^{\ast \ast }(x_0)<f(x_0)$ at some $x_0$. The epigraph $\mathrm{epi}f$ is a closed convex subset of ${\mathbb{R}}^{d+1}$ because $f$ is convex and lsc. The point $(x_0,f^{\ast \ast }(x_0))$ lies strictly below $\mathrm{epi}f$, hence is not in it. The second geometric Hahn-Banach theorem 02.11.02 separates the compact convex point $\{(x_0,f^{\ast \ast }(x_0))\}$ from the closed convex set $\mathrm{epi}f$ strictly: there is a non-zero $(\mu ,s)\in {\mathbb{R}}^d\times \mathbb{R}$ and $\alpha \in \mathbb{R}$ with

$$⟨\mu ,x_0⟩+s{f}^{\ast \ast }(x_0)<\alpha \le ⟨\mu ,x⟩+st\text{for all }(x,t)\in \mathrm{epi}f.$$

Because $t$ ranges up to $+\infty$ on the epigraph, $s\ge 0$. First take $s>0$; rescale so $s=1$ and set $\lambda :=-\mu$. Evaluating the right inequality at $t=f(x)$ gives $⟨\mu ,x⟩+f(x)\ge \alpha$, i.e. $⟨\lambda ,x⟩-f(x)\le -\alpha$ for all $x$, so $f^{\ast }(\lambda )\le -\alpha$. Then

$$f^{\ast \ast }(x_0)\ge ⟨\lambda ,x_0⟩-f^{\ast }(\lambda )\ge ⟨\lambda ,x_0⟩+\alpha =-⟨\mu ,x_0⟩+\alpha >f^{\ast \ast }(x_0),$$

a contradiction. The boundary case $s=0$ (a vertical separating hyperplane) forces $⟨\mu ,x_0⟩<\alpha \le ⟨\mu ,x⟩$ for all $x\in \mathrm{dom}f$; combined with any ${\lambda }_0$ for which $f^{\ast }({\lambda }_0)<\infty$ (one exists because $f$ is proper, by an application of (i) at any subgradient point or a recession argument), the functionals ${\lambda }_0+n\mu$ drive $⟨{\lambda }_0+n\mu ,x_0⟩-f^{\ast }({\lambda }_0+n\mu )\to \infty$ as $n\to \infty$, so $f^{\ast \ast }(x_0)=+\infty \ge f(x_0)$, again contradicting $f^{\ast \ast }(x_0)<f(x_0)$. Hence $f^{\ast \ast }=f$. $\square$

Bridge. This theorem builds toward the entire large-deviations dictionary and appears again in the analysis of every rate function. This is exactly the closed-convex involution $f\leftrightarrow f^{\ast }$ that generalises the smooth, invertible Legendre transform of 09.04.01 and 11.01.02: where mechanics required $f$ to be a $C^2$ Legendre function so that $\nabla f$ inverts, biconjugation needs only closedness and convexity, recovering the smooth case when $f$ is essentially smooth and essentially strictly convex. The foundational reason the cumulant generating function and the rate function are interchangeable is that one is the Fenchel conjugate of the other, and putting these together with the Fenchel-Young equality condition $\lambda \in \partial f(x)$ pins down the optimal exponential tilt that realises a given rare value. The supporting-hyperplane step is dual to the subgradient existence guaranteed by Hahn-Banach separation 02.11.02.

Exercises Intermediate+

Advanced results Master

Conjugacy of subdifferentials and the duality of differentiability with strict convexity

For a closed proper convex $f$, the Fenchel-Young equality condition promotes to an inversion of set-valued maps: $\lambda \in \partial f(x)$ if and only if $x\in \partial f^{\ast }(\lambda )$, so $\partial f^{\ast }=(\partial f{)}^{-1}$ as relations on ${\mathbb{R}}^d\times {\mathbb{R}}^d$. Single-valuedness of $\partial f$ at $x$ (differentiability of $f$) corresponds to $\partial f^{\ast }$ being constant on an exposed face (strict convexity of $f^{\ast }$ along that direction), and conversely. This is the precise sense in which differentiability of $f$ is dual to strict convexity of $f^{\ast }$. When $f$ is a Legendre function in the sense of ^{[Rockafellar §26]}, $\partial f=\{\nabla f\}$ everywhere on $\mathrm{int}(\mathrm{dom}f)$, the inverse map is $\nabla f^{\ast }=(\nabla f{)}^{-1}$, and the smooth Legendre transform of mechanics 09.04.01 and thermodynamics 11.01.02 is recovered as the differentiable core of the general theory.

Cramér's theorem: the conjugate as rate function

Let $X_1,X_2,\dots$ be i.i.d. in ${\mathbb{R}}^d$ with cumulant generating function $\Lambda (\lambda )=\log \mathbb{E}{e}^{⟨\lambda ,X_1⟩}$, finite in a neighbourhood of the origin. Cramér's theorem ^{[Cramér 1938]} asserts that the empirical means $S_n/n=(X_1+\cdots +X_n)/n$ satisfy a large deviation principle with good rate function ${\Lambda }^{\ast }$: for closed $F$ and open $G$,

$$\underset{n}{\mathrm{limsup}}\frac{1}{n}\log \mathbb{P}(S_n/n\in F)\le -\underset{F}{\inf }{\Lambda }^{\ast },\underset{n}{\mathrm{liminf}}\frac{1}{n}\log \mathbb{P}(S_n/n\in G)\ge -\underset{G}{\inf }{\Lambda }^{\ast }.$$

The upper bound is the Chernoff argument: for any $\lambda$, Markov's inequality on $e^{⟨\lambda ,S_n⟩}$ gives $\mathbb{P}(S_n/n\in F)\le e^{-n(⟨\lambda ,x⟩-\Lambda (\lambda ))}$ uniformly, and optimising over $\lambda$ produces the conjugate ${\Lambda }^{\ast }$. The lower bound uses the exponential change of measure $d{\mathbb{P}}_{\lambda }/d\mathbb{P}=e^{⟨\lambda ,X⟩-\Lambda (\lambda )}$, the tilted law, whose mean is $\nabla \Lambda (\lambda )$; the Fenchel-Young equality ${\Lambda }^{\ast }(x)=⟨{\lambda }_x,x⟩-\Lambda ({\lambda }_x)$ at the tilt ${\lambda }_x$ solving $\nabla \Lambda ({\lambda }_x)=x$ identifies the cost. That ${\Lambda }^{\ast }$ is a good rate function is Exercise 7.

The Gärtner-Ellis theorem and steepness

When the $X_i$ are not i.i.d. but the scaled limit $\Lambda (\lambda ):={\lim }_n\frac{1}{n}\log \mathbb{E}{e}^{⟨\lambda ,S_n⟩}$ exists as a lsc convex function that is essentially smooth with $0\in \mathrm{int}(\mathrm{dom}\Lambda )$, the Gärtner-Ellis theorem ^{[Dembo & Zeitouni §2.3]} gives the large deviation principle with rate function ${\Lambda }^{\ast }$. Steepness is the load-bearing hypothesis: it guarantees that every $x$ in the interior of $\mathrm{dom}{\Lambda }^{\ast }$ is exposed, i.e. realised as $\nabla \Lambda ({\lambda }_x)$ for some ${\lambda }_x$, so the lower bound's tilt exists. Without steepness, exposed points may be missing and the rate function can fail to be the conjugate at non-exposed boundary points — the same phenomenon visible in Exercise 8, now with probabilistic teeth.

Varadhan's lemma and the contraction principle

Two consequences extend the dictionary. Varadhan's lemma evaluates exponential integrals: if $\{S_n/n\}$ satisfies an LDP with good rate $I$ and $\varphi$ is continuous with a mild tail condition, then $\frac{1}{n}\log \mathbb{E}{e}^{n\varphi (S_n/n)}\to {\sup }_x(\varphi (x)-I(x))$, a Fenchel-type variational formula generalising the Laplace method. The contraction principle transports an LDP through a continuous map $T$: the pushed-forward rate function is $I_T(y)=\inf \{I(x):T(x)=y\}$, an infimal projection that is conjugacy-compatible — conjugation turns infimal projections into restrictions, so $(I_T{)}^{\ast }=I^{\ast }\circ T^{\ast }$ in the linear case. The Legendre-Fenchel transform threads through all of these as the operation converting the multiplicative, moment-side description into the additive, cost-side description.

Synthesis. The Legendre-Fenchel transform is exactly the involution that generalises the smooth Legendre transform of mechanics 09.04.01 and thermodynamics 11.01.02 to the closed-convex category, and the central insight of large-deviation theory is that the cumulant generating function $\Lambda$ and the rate function ${\Lambda }^{\ast }$ are a conjugate pair. The foundational reason Cramér's theorem holds is that the Chernoff upper bound optimised over exponential tilts is the supremum defining ${\Lambda }^{\ast }$, while the lower bound's change of measure realises the Fenchel-Young equality at the optimal tilt; putting these together, steepness and essential smoothness control exactly which points are exposed and hence where the conjugate is sharp, the same regularity that makes the mechanical transform a diffeomorphism. The construction is dual to the smooth case in the literal Fenchel sense, appears again in statistical mechanics where ${\Lambda }^{\ast }$ is the entropy and $\Lambda$ the free energy, and builds toward the full Cramér 37.07.01, Sanov, and Gärtner-Ellis 37.07.04 apparatus. The bridge is biconjugation: ${\Lambda }^{\ast \ast }=\Lambda$ on the convex closure is what lets one pass freely between the rare-event cost and the moment generating data.

Full proof set Master

Proposition 1 (conjugacy inverts subdifferentials). Let $f$ be closed proper convex. Then $\lambda \in \partial f(x) \iff x \in \partial f^(\lambda) \iff f(x) + f^(\lambda) = \langle\lambda, x\rangle$.

Proof. The Fenchel-Young theorem (Key theorem, part (i)) gives $\lambda \in \partial f(x)⟺f(x)+f^{\ast }(\lambda )=⟨\lambda ,x⟩$. By Fenchel-Moreau (part (ii)), $f=f^{\ast \ast }$, so applying the same equivalence to $f^{\ast }$ in place of $f$ yields $x\in \partial f^{\ast }(\lambda )⟺f^{\ast }(\lambda )+f^{\ast \ast }(x)=⟨\lambda ,x⟩⟺f^{\ast }(\lambda )+f(x)=⟨\lambda ,x⟩$. The three conditions coincide. $\square$

Proposition 2 ($\Lambda$ is convex and ${\Lambda }^{\ast }$ is a good rate function). Let $X$ be an ${\mathbb{R}}^d$-valued random vector with $\Lambda (\lambda )=\log \mathbb{E}{e}^{⟨\lambda ,X⟩}<\infty$ for $\lambda$ in an open neighbourhood $N$ of $0$. Then $\Lambda$ is convex on $N$, $\Lambda (0)=0$, and $\Lambda^$ is non-negative, lsc, convex, with compact sublevel sets.*

Proof. Convexity is the Hölder argument of Exercise 6 applied coordinatewise to the pairing $⟨\lambda ,X⟩$: for $\theta \in [0,1]$ and ${\lambda }_1,{\lambda }_2\in N$, $\mathbb{E}{e}^{⟨\theta {\lambda }_1+(1-\theta ){\lambda }_2,X⟩}\le (\mathbb{E}{e}^{⟨{\lambda }_1,X⟩}{)}^{\theta }(\mathbb{E}{e}^{⟨{\lambda }_2,X⟩}{)}^{1-\theta }$ by Hölder with exponents $1/\theta ,1/(1-\theta )$; take logarithms. $\Lambda (0)=\log \mathbb{E}1=0$. Non-negativity of ${\Lambda }^{\ast }$: ${\Lambda }^{\ast }(x)\ge ⟨0,x⟩-\Lambda (0)=0$. Convexity and lower-semicontinuity of ${\Lambda }^{\ast }$ hold for any conjugate (Exercise 5). For compact sublevels, pick $\delta >0$ with the closed ball $\bar{B}(0,\delta )\subset N$ and set $M={\max }_{\Vert \lambda \Vert \le \delta }\Lambda (\lambda )<\infty$ (a maximum of a continuous convex function on a compact set). For each $x$ choose $\lambda =\delta x/\Vert x\Vert$ (for $x\ne 0$): ${\Lambda }^{\ast }(x)\ge ⟨\lambda ,x⟩-\Lambda (\lambda )\ge \delta \Vert x\Vert -M$. Hence $\{{\Lambda }^{\ast }\le \alpha \}\subseteq \{\Vert x\Vert \le (\alpha +M)/\delta \}$, bounded; closed by lsc; therefore compact. $\square$

Proposition 3 (Chernoff upper bound). With $\Lambda$ as above and $S_n=X_1+\cdots +X_n$ for i.i.d. copies, for any closed half-space $H=\{x:⟨\lambda ,x⟩\ge ⟨\lambda ,a⟩\}$ with $\lambda$ in the domain of $\Lambda$,

$$\frac{1}{n}\log \mathbb{P}(S_n/n\in H)\le -(⟨\lambda ,a⟩-\Lambda (\lambda )).$$

Proof. By Markov's inequality applied to the non-negative variable $e^{n⟨\lambda ,S_n/n⟩}$,

$$\mathbb{P}(S_n/n\in H)=\mathbb{P}(⟨\lambda ,S_n⟩\ge n⟨\lambda ,a⟩)\le e^{-n⟨\lambda ,a⟩}\mathbb{E}{e}^{⟨\lambda ,S_n⟩}=e^{-n⟨\lambda ,a⟩}(\mathbb{E}{e}^{⟨\lambda ,X_1⟩}{)}^n,$$

using independence in the last step. Taking $\frac{1}{n}\log$ gives $\frac{1}{n}\log \mathbb{P}(S_n/n\in H)\le -⟨\lambda ,a⟩+\Lambda (\lambda )$. Optimising the bound over admissible $\lambda$ and over half-spaces containing a closed set $F$ recovers $-{\inf }_{x\in F}{\Lambda }^{\ast }(x)$, the Cramér upper bound. $\square$

Connections Master

• The smooth, invertible Legendre transform of Lagrangian-to-Hamiltonian mechanics 09.04.01 is the special case of this transform restricted to $C^2$ essentially smooth, essentially strictly convex (Legendre) functions, where $\nabla f$ is a diffeomorphism and the supremum is attained at the unique stationary point; the general lsc-convex theory developed here is what that unit's "Legendre-Fenchel" coda gestures at without proving.

• The thermodynamic potentials of 11.01.02 — Helmholtz and Gibbs free energies, enthalpy — are Legendre-Fenchel conjugates of the internal energy, and the convex-duality structure here is exactly the Legendre-duality between entropy and free energy that statistical mechanics makes precise; the rate function ${\Lambda }^{\ast }$ of large deviations is the thermodynamic entropy and $\Lambda$ the scaled free energy.

• The biconjugation theorem $f^{\ast \ast }=f$ rests on the geometric Hahn-Banach separation of a point from the closed convex epigraph proved in 02.11.02; that unit's subdifferential and Fenchel-Moreau material is the functional-analytic engine, specialised here from a general normed space to ${\mathbb{R}}^d$ with the additional probabilistic payload.

• Cramér's theorem 37.07.01 takes the conjugate ${\Lambda }^{\ast }$ constructed here as its rate function, and the Gärtner-Ellis theorem 37.07.04 extends the identification to non-i.i.d. arrays under the steepness hypothesis whose meaning is pinned down in this unit's essential-smoothness discussion.

Historical & philosophical context Master

The one-variable Legendre transformation dates to Legendre's work on differential equations and the theory of surfaces in the 1780s, and entered mechanics through Hamilton (1834) and thermodynamics through Massieu (1869) and Gibbs (1873). Its convex-analytic generalisation is due to Werner Fenchel, who in On conjugate convex functions ^{[Fenchel 1949]} (Canadian Journal of Mathematics 1, 73-77) defined the conjugate of a general convex function via the supremum formula and proved the biconjugation duality without any smoothness hypothesis, replacing the requirement that $\nabla f$ be invertible by supporting-hyperplane arguments. Jean-Jacques Moreau extended the theory to infinite-dimensional spaces in the early 1960s, and R. Tyrrell Rockafellar's Convex Analysis ^{[Rockafellar §12, §26]} (Princeton, 1970) gave the now-standard systematic account, including the Legendre-function characterisation that locates the classical smooth transform inside the general framework.

The probabilistic significance traces to Harald Cramér's 1938 paper ^{[Cramér 1938]} (Actualités Scientifiques et Industrielles 736), which computed the exponential decay rate of large deviations of sums of i.i.d. variables and identified the rate as the conjugate of the cumulant generating function, although Cramér worked under analytic-density assumptions later removed. The general formulation as a large deviation principle is due to Varadhan (1966) and was placed in its convex-duality setting by Lanford, Ellis, Gärtner, and others through the 1970s; Dembo and Zeitouni ^{[Dembo & Zeitouni §2.2]} systematised the i.i.d. and Gärtner-Ellis routes. The identification of ${\Lambda }^{\ast }$ with thermodynamic entropy and $\Lambda$ with free energy was made explicit by Ellis (1985) and recast for physicists by Touchette (2009).

Bibliography Master

@article{fenchel1949conjugate,
  author  = {Fenchel, Werner},
  title   = {On conjugate convex functions},
  journal = {Canadian Journal of Mathematics},
  volume  = {1},
  pages   = {73--77},
  year    = {1949}
}

@book{rockafellar1970convex,
  author    = {Rockafellar, R. Tyrrell},
  title     = {Convex Analysis},
  series    = {Princeton Mathematical Series},
  number    = {28},
  publisher = {Princeton University Press},
  year      = {1970}
}

@book{dembozeitouni1998ldp,
  author    = {Dembo, Amir and Zeitouni, Ofer},
  title     = {Large Deviations Techniques and Applications},
  edition   = {2nd},
  series    = {Applications of Mathematics},
  number    = {38},
  publisher = {Springer},
  year      = {1998}
}

@article{cramer1938nouveau,
  author  = {Cram\'er, Harald},
  title   = {Sur un nouveau th\'eor\`eme-limite de la th\'eorie des probabilit\'es},
  journal = {Actualit\'es Scientifiques et Industrielles},
  volume  = {736},
  pages   = {5--23},
  year    = {1938}
}

@book{ellis1985entropy,
  author    = {Ellis, Richard S.},
  title     = {Entropy, Large Deviations, and Statistical Mechanics},
  series    = {Grundlehren der mathematischen Wissenschaften},
  number    = {271},
  publisher = {Springer},
  year      = {1985}
}

@article{touchette2009large,
  author  = {Touchette, Hugo},
  title   = {The large deviation approach to statistical mechanics},
  journal = {Physics Reports},
  volume  = {478},
  number  = {1--3},
  pages   = {1--69},
  year    = {2009}
}

@book{hiriarturruty2001fundamentals,
  author    = {Hiriart-Urruty, Jean-Baptiste and Lemar\'echal, Claude},
  title     = {Fundamentals of Convex Analysis},
  series    = {Grundlehren Text Editions},
  publisher = {Springer},
  year      = {2001}
}