The Convolution Theorem and the Local Asymptotic Minimax Bound

Intuition Beginner

Once a smooth model has collapsed into a single Gaussian measurement of the unknown shift, you can ask the sharpest question in all of estimation: among every sensible way of guessing that shift, which is best, and how good can the best one be? This unit answers it. The surprising news is that there is one champion, the same efficient guess that the maximum-likelihood method produces, and every other sensible guess is that champion with extra noise stirred in.

Picture the efficient guess as a clean bell curve with a fixed spread, the spread set by the model's sharpness (the Fisher information). Now take any rival guess that plays fair — meaning it does not secretly tune itself to one favoured value of the truth. Its bell curve is always the champion's bell curve blurred by independent extra noise. Adding independent noise only widens a bell curve; it can never narrow it. So no fair rival is ever tighter than the champion. That single fact is the convolution theorem.

There is a sneaky way to look tighter at one special point — the Hodges trick — but it pays for the trick by being terrible just next door. A second bound, the local minimax bound, looks at the worst case over a whole tiny neighbourhood and shows the champion's spread is the floor there too, so the trick buys nothing once you stop cherry-picking.

The takeaway: in the Gaussian limit the efficient estimator is the unique best, every fair estimator is it plus independent noise, and no estimator beats its spread over a shrinking neighbourhood; that is the exact meaning of "asymptotically efficient."

Visual Beginner

Picture two bell curves drawn on the same axis. The narrow one, drawn solid, is the efficient estimator's error: a bell centred at zero with spread one over the Fisher information. The wider one, drawn dashed, is some fair rival's error. The lesson of the convolution theorem is that the dashed curve is the solid curve with an independent puff of noise added — and adding noise only ever spreads a bell wider, never tighter. So the dashed curve hugs the solid one or sits outside it, never inside.

question	efficient estimator	any fair rival
shape of the error	one bell curve	the same bell plus extra noise
spread	one over the Fisher information	at least that, usually more
how the two relate	the champion	champion convolved with independent noise
can the rival be tighter	—	no, convolution only widens

The takeaway: every fair estimator's error is the efficient error blurred by independent noise, so the efficient spread one over the Fisher information is the floor nobody beats.

Worked example Beginner

We watch the convolution bound bite for a normal mean. Each observation is normal with variance one and unknown mean ${\theta }_0$. The Fisher information per observation is $1$, so the efficient large-sample error of the sample average has spread (variance) $1/1=1$ in the rescaled units.

Step 1. The champion. The sample average ${\bar{X}}_n$ is the efficient estimator. Its rescaled error $\sqrt{n}({\bar{X}}_n-{\theta }_0)$ has variance $1$ — a clean bell, variance $1$, no extra noise.

Step 2. A fair rival. Consider the estimator that averages the data but then adds a tiny independent random nudge with variance $0.3$, the nudge drawn fresh and unrelated to the data. Its rescaled error is the champion's error plus that independent nudge.

Step 3. Add the spreads. Independent noises add their variances: the rival's error variance is $1+0.3=1.3$. The rival is fair (its blur does not depend on the true mean), and its spread is larger than the champion's $1$.

Step 4. Check the floor. Could a fair rival have variance below $1$? No. Its variance is always $1$ plus the variance of some independent extra piece, and a variance is never negative, so the total is at least $1$. The floor is exactly the efficient variance.

What this tells us: adding independent noise to the efficient estimator can only raise the variance above $1$, never lower it, so the efficient variance one over the Fisher information is the best any fair estimator can do.

Check your understanding Beginner

Formal definition Intermediate+

Let ${\mathcal{E}}_n=({\mathcal{X}}^{(n)},{\mathcal{A}}^{(n)},\{P_{\theta ,n}:\theta \in \Theta \})$, $\Theta \subseteq {\mathbb{R}}^k$ open, be a sequence of experiments that is LAN at an interior ${\theta }_0$ with rate $\sqrt{n}$, central sequence ${\Delta }_{n,{\theta }_0}\stackrel{d}{\to }N(0,I({\theta }_0))$, and Fisher information $I({\theta }_0)\succ 0$, in the sense of 45.04.06. Convergence $\stackrel{d}{\to }$, $\stackrel{\mathbb{P}}{\to }$, and the $o_{\mathbb{P}}$ calculus are as in 45.04.01; contiguity and Le Cam's third lemma are as in 45.04.07.

Definition (regular estimator). A sequence ${\hat{\theta }}_n$ of estimators of $\theta$ is regular at ${\theta }_0$ (for the rate $\sqrt{n}$) if there is a fixed probability law $L$ on ${\mathbb{R}}^k$, not depending on $h$, such that for every $h\in {\mathbb{R}}^k$,

$$\sqrt{n}({\hat{\theta }}_n-{\theta }_0-h/\sqrt{n})\stackrel{d}{\to }L\text{under }{P}_{{\theta }_0+h/\sqrt{n},n}.$$

Regularity is local shift-equivariance of the limit law: when the truth is moved by the local amount $h/\sqrt{n}$, the estimator's centred error has the same limit. It rules out estimators whose performance jumps as the truth crosses a distinguished value.

Definition (bowl-shaped loss). A loss $\ell :{\mathbb{R}}^k\to [0,\infty )$ is bowl-shaped (subconvex) if its sublevel sets $\{x:\ell (x)\le c\}$ are convex and symmetric about the origin for every $c\ge 0$. Squared error $\ell (x)=x^{\mathsf{T}}Ax$ with $A⪰0$, absolute error, and all indicators of complements of symmetric convex sets are bowl-shaped.

Definition (Gaussian shift risk and the convolution class). In the Gaussian shift experiment ${\mathcal{G}}_I=\{N(h,I^{-1}):h\in {\mathbb{R}}^k\}$ of 45.04.06, an estimator $T(\Delta )$ of $h$ based on the single observation $\Delta \sim N(Ih,I)$ is shift-equivariant if $T(\Delta +Ic)=T(\Delta )+c$ for all $c$. The efficient estimator is $\hat{h}=I^{-1}\Delta \sim N(h,I^{-1})$. A law $L$ on ${\mathbb{R}}^k$ is said to lie in the convolution class of ${\theta }_0$ if $L=N(0,I^{-1})\ast M$ for some probability law $M$.

Definition (pathwise-differentiable functional and efficient influence function). Let $\psi (P)\in \mathbb{R}$ be a functional on a (possibly nonparametric) model $\mathcal{P}$ through $P_{{\theta }_0}$. The tangent set ${\dot{\mathcal{P}}}_{{\theta }_0}$ is the collection of scores $g\in L_0^2(P_{{\theta }_0})$ (mean-zero, square-integrable) of one-dimensional regular submodels $t\mapsto P_t$ with $P_0=P_{{\theta }_0}$; its closed linear span is the tangent space $\overline{\mathrm{lin}}{\dot{\mathcal{P}}}_{{\theta }_0}$. The functional is pathwise differentiable at $P_{{\theta }_0}$ if there is $\tilde{\psi }\in L_0^2(P_{{\theta }_0})$, a gradient, with $\frac{d}{dt}\psi (P_t){|}_{t=0}={\mathbb{E}}_{{\theta }_0}[\tilde{\psi }g]$ for every submodel score $g$. The efficient influence function ${\tilde{\psi }}_{\text{eff}}$ is the orthogonal projection of any gradient onto the tangent space, and the efficiency bound (information bound) for $\psi$ is ${\mathbb{E}}_{{\theta }_0}{\tilde{\psi }}_{\text{eff}}^2$.

Counterexamples to common slips Intermediate+

• The convolution theorem is about regular estimators, and the restriction is essential. The Hodges estimator ${\hat{\theta }}_n={\bar{X}}_{n}1\{|{\bar{X}}_n|>n^{-1/4}\}$ in $N(\theta ,1)$ has limit law $N(0,1)$ for ${\theta }_0\ne 0$ but the degenerate ${\delta }_0$ at ${\theta }_0=0$ — variance $0<1$ at one point. It is not regular ($L$ depends on $h$ at ${\theta }_0=0$), so it is no counterexample; the LAM bound shows its maximal local risk near $0$ blows up.

• Convolution gives a Loewner-order bound, not just a determinant bound. The conclusion is $\mathrm{Cov}(L)=I^{-1}+\mathrm{Cov}(M)⪰I^{-1}$ in the positive-semidefinite order, so every linear functional of the estimator is at least as variable as the efficient one. A common slip weakens this to a comparison of total variances or generalised variances.

• The minimax bound needs the supremum over neighbourhoods, taken before the liminf. Writing ${\mathrm{liminf}}_n{\sup }_h\mathbb{E}\ell$ with the supremum over a fixed bounded $h$-set, then enlarging, is the correct order; swapping to ${\sup }_h{\mathrm{liminf}}_n$ would let a superefficient point escape the bound. The LAM statement takes the supremum over finite $I$ outside the liminf.

• The efficient influence function is a projection, not the gradient itself. Any gradient $\tilde{\psi }$ has $\mathbb{E}[\tilde{\psi }g]=\dot{\psi }(g)$ for tangent $g$, but only the component lying in the tangent space is attainable; the orthogonal complement inflates variance without helping. In a correctly specified parametric model the tangent space is all of ${\mathbb{R}}^k$-spanned scores and the efficient influence function is $\dot{\psi }({\theta }_0{)}^{\mathsf{T}}{I}^{-1}{\dot{\ell }}_{{\theta }_0}$.

Key theorem with proof Intermediate+

The signature result is Hájek's convolution theorem: in a LAN family the limit law of any regular estimator is the efficient Gaussian law convolved with an independent law, so the efficient estimator is best and any regular competitor is at least as dispersed. The proof is read off the Gaussian shift limit experiment. Regularity plus Le Cam's third lemma 45.04.07 forces the estimator's joint limit with the central sequence to have a specific cross-covariance; that cross-covariance is exactly the identity, which makes the residual independent of the efficient part, and the factorisation is the convolution. The argument lives entirely in the limit experiment, where it becomes a statement about shift-equivariant estimators of a Gaussian mean.

Theorem (Hájek's convolution theorem; van der Vaart Ch. 8). Let the sequence be LAN at ${\theta }_0$ with information $I=I({\theta }_0)\succ 0$ and central sequence ${\Delta }_{n,{\theta }_0}\stackrel{d}{\to }\Delta \sim N(0,I)$. Let ${\hat{\theta }}_n$ be a regular estimator with limit law $L$. Then there is a probability law $M$ on ${\mathbb{R}}^k$ with

$$L=N(0,I^{-1})\ast M.$$

Consequently $\mathrm{Cov}(L)⪰I^{-1}$ in the Loewner order, with equality iff $M={\delta }_0$, i.e. iff ${\hat{\theta }}_n$ is asymptotically equivalent to the efficient estimator ${\theta }_0+n^{-1/2}{I}^{-1}{\Delta }_{n,{\theta }_0}$.

Proof. Write $S_n=\sqrt{n}({\hat{\theta }}_n-{\theta }_0)$ and abbreviate ${\Delta }_n={\Delta }_{n,{\theta }_0}$. By LAN the local log-likelihood ratio for the alternative ${\theta }_0+h/\sqrt{n}$ is ${\Lambda }_n(h)=⟨h,{\Delta }_n⟩-\frac{1}{2}⟨h,Ih⟩+o_{P_{{\theta }_0}}(1)$.

Step 1: joint null limit. Pass to a subsequence along which $(S_n,{\Delta }_n)$ converges jointly under $P_{{\theta }_0,n}$, to a limit $(S,\Delta )$ with $\Delta \sim N(0,I)$. By Prohorov the joint family is tight: ${\Delta }_n$ converges and $S_n$ is tight because ${\hat{\theta }}_n$ is regular (the $h=0$ case gives $S_n\stackrel{d}{\to }L$). Then $(S_n,{\Lambda }_n(h))$ converges jointly to $(S,⟨h,\Delta ⟩-\frac{1}{2}⟨h,Ih⟩)$ by the continuous-mapping theorem applied to the affine map in ${\Delta }_n$ and Slutsky for the $o_P(1)$ term.

Step 2: the third lemma fixes the cross-covariance. By Le Cam's third lemma 45.04.07, under $P_{{\theta }_0+h/\sqrt{n},n}$ the statistic $S_n$ has limit law equal to the null law of $S$ shifted in mean by $\mathrm{Cov}(S,⟨h,\Delta ⟩)=\mathrm{Cov}(S,\Delta )h$. Regularity asserts that under this same alternative $S_n-h\stackrel{d}{\to }L$, i.e. $S_n\stackrel{d}{\to }L\ast {\delta }_h$, the null law of $S$ shifted by exactly $h$. Equating the two shifts for all $h$ forces

$$\mathrm{Cov}(S,\Delta )h=h\text{for all }h,\text{hence}\mathrm{Cov}(S,\Delta )={\mathrm{Id}}_k.$$

Step 3: decompose into efficient part and independent residual. Set $W=S-I^{-1}\Delta$. Then

$$\mathrm{Cov}(W,\Delta )=\mathrm{Cov}(S,\Delta )-I^{-1}\mathrm{Cov}(\Delta ,\Delta )={\mathrm{Id}}_k-I^{-1}I=0,$$

so $W$ is uncorrelated with the Gaussian $\Delta$. The limit objects live in the Gaussian shift experiment, where the joint law of $(W,\Delta )$ is stable under the Gaussian exponential tilt of the third lemma; uncorrelatedness of $W$ with the Gaussian generator $\Delta$ under that tilt-stability upgrades to independence. (Concretely: regularity makes the conditional law of $W$ given $\Delta$ invariant under the shift, which for a jointly Gaussian-tilt-stable family means $W\perp \Delta$.)

Step 4: read off the convolution. Now $S=I^{-1}\Delta +W$ with $I^{-1}\Delta \sim N(0,I^{-1})$ and $W\perp \Delta$. Therefore

$$L=\mathcal{L}(S)=\mathcal{L}(I^{-1}\Delta )\ast \mathcal{L}(W)=N(0,I^{-1})\ast M,M:=\mathcal{L}(W).$$

The covariance is $\mathrm{Cov}(L)=I^{-1}+\mathrm{Cov}(M)⪰I^{-1}$, with equality iff $\mathrm{Cov}(M)=0$, i.e. $M={\delta }_0$ and $S=I^{-1}\Delta$. Since the subsequential limit $L$ does not depend on the subsequence (it is the fixed regular limit), the factorisation holds along the full sequence. $\square$

Bridge. This theorem builds toward the entire optimality theory of asymptotic estimation, and the same Gaussian-shift argument appears again in the local asymptotic minimax bound below, where the convolution structure is replaced by a minimax computation over the neighbourhood but the floor $N(0,I^{-1})$ is identical. The foundational reason the efficient variance $I^{-1}$ is unbeatable is that the local experiment is, to second order, a single Gaussian observation of the shift 45.04.06, and in that experiment shift-equivariance forces the cross-covariance with the sufficient statistic $\Delta$ to be the identity — this is exactly the mechanism that pins the MLE's asymptotic variance to $I^{-1}$ in 45.04.03, now stated as a bound over all regular competitors rather than a property of one estimator. The convolution is dual to the Cramér-Rao inequality of 45.01.05: the finite-sample lower bound on variance reappears as a distributional factorisation in which the efficient law is a factor of every regular law. Putting these together, the bridge is that asymptotic efficiency means being the unconvolved Gaussian factor: an estimator is efficient precisely when its residual $M$ collapses to a point mass, and the central insight is that no regular estimator can erase the $N(0,I^{-1})$ factor, only add noise to it.

Exercises Intermediate+

Advanced results Master

The efficiency theory is read off the Gaussian shift limit experiment: regularity converts a sequence of estimators into a shift-equivariant procedure in $\{N(h,I^{-1})\}$, where the convolution and minimax bounds are exact finite-dimensional facts transported back by the asymptotic-representation theorem. The convolution theorem bounds regular estimators by a distributional factorisation; the local asymptotic minimax theorem removes regularity by a worst-case-over-neighbourhoods criterion; the Anderson lemma supplies the peakedness that makes the centred Gaussian optimal for every bowl-shaped loss; and the semiparametric extension replaces the finite-dimensional information $I^{-1}$ by the squared norm of the efficient influence function, the projection of a gradient onto the model's tangent space.

Theorem 1 (asymptotic-representation theorem). Under LAN at ${\theta }_0$, for any sequence of statistics $T_n$ with $\sqrt{n}$-localised limits there is a (possibly randomised) statistic $T$ in the Gaussian shift experiment $\{N(h,I^{-1})\}$ such that, for every $h$, the limit under $P_{{\theta }_0+h/\sqrt{n},n}$ of $\mathcal{L}(T_n)$ equals ${\mathcal{L}}_h(T)$. Every asymptotic risk of $T_n$ is matched by the risk of $T$ in the limit experiment, so lower bounds proved in $\{N(h,I^{-1})\}$ transfer verbatim to the original sequence ^{[Le Cam, L. — Asymptotic Methods in Statistical Decision Theory]}.

Theorem 2 (Hájek's convolution theorem). In a LAN sequence at ${\theta }_0$, the limit law $L$ of any regular estimator satisfies $L=N(0,I({\theta }_0{)}^{-1})\ast M$ for a probability law $M$; hence $\mathrm{Cov}(L)⪰I({\theta }_0{)}^{-1}$ in the Loewner order, with equality iff $M={\delta }_0$, the efficient case where ${\hat{\theta }}_n={\theta }_0+n^{-1/2}{I}^{-1}{\Delta }_{n,{\theta }_0}+o_P(n^{-1/2})$. This is the asymptotic incarnation of the Cramér-Rao bound of 45.01.05 and the optimality benchmark for the MLE of 45.04.03, now stated for the whole regular class ^{[Hájek, J. — A characterization of limiting distributions of regular estimates]}.

Theorem 3 (Hájek-Le Cam local asymptotic minimax). For any estimator sequence ${\hat{\theta }}_n$ and any bowl-shaped loss $\ell$,

$$\underset{I\text{ finite}}{\sup }\underset{n}{\mathrm{liminf}}\underset{h\in I}{\sup }{\mathbb{E}}_{{\theta }_0+h/\sqrt{n}}\ell (\sqrt{n}({\hat{\theta }}_n-{\theta }_0-h/\sqrt{n}))\ge \mathbb{E}\ell (Z),Z\sim N(0,I({\theta }_0{)}^{-1}),$$

the supremum over finite subsets $I\subset {\mathbb{R}}^k$ of local parameters. The Gaussian shift attains the bound with the efficient estimator $T=\Delta$, so no estimator beats $N(0,I^{-1})$ uniformly over shrinking neighbourhoods. Unlike the convolution theorem this requires no regularity, which is why it dissolves the superefficiency of 45.04.03: a pointwise gain is paid for by inflated local-minimax risk nearby ^{[Hájek, J. — Local asymptotic minimax and admissibility in estimation]}.

Theorem 4 (Anderson's lemma). Let $Z$ have a symmetric, unimodal (quasi-concave density) law on ${\mathbb{R}}^k$ — in particular $Z\sim N(0,\Sigma )$ — and let $C\subset {\mathbb{R}}^k$ be symmetric and convex. Then $\mathbb{P}(Z+v\in C)\le \mathbb{P}(Z\in C)$ for every $v$, with the consequence that $\mathbb{E}\ell (Z+v)\ge \mathbb{E}\ell (Z)$ for every bowl-shaped loss $\ell$. This peakedness is the analytic mechanism by which both the convolution residual $M$ and the minimax shift can only increase risk ^{[van der Vaart — Asymptotic Statistics]}.

Theorem 5 (semiparametric efficiency bound). Let $\psi :\mathcal{P}\to \mathbb{R}$ be pathwise differentiable at $P_{{\theta }_0}$ with tangent space $\mathcal{T}=\overline{\mathrm{lin}}{\dot{\mathcal{P}}}_{{\theta }_0}\subseteq L_0^2(P_{{\theta }_0})$ and a gradient $\tilde{\psi }$. The efficient influence function ${\tilde{\psi }}_{\text{eff}}=\Pi (\tilde{\psi }\mid \mathcal{T})$ is the orthogonal projection of any gradient onto $\mathcal{T}$ (independent of the chosen gradient, since two gradients differ by an element of ${\mathcal{T}}^{\perp }$), and any regular estimator ${\hat{\psi }}_n$ of $\psi$ has $\sqrt{n}({\hat{\psi }}_n-\psi (P_{{\theta }_0}))\stackrel{d}{\to }N(0,V)\ast M$ with $V\ge {\mathbb{E}}_{{\theta }_0}{\tilde{\psi }}_{\text{eff}}^2$, the efficiency bound. An estimator is asymptotically efficient iff it is asymptotically linear with influence function ${\tilde{\psi }}_{\text{eff}}$: $\sqrt{n}({\hat{\psi }}_n-\psi )=n^{-1/2}{\sum }_i{\tilde{\psi }}_{\text{eff}}(X_i)+o_P(1)$ ^{[Bickel, P. J., Klaassen, C. A. J., Ritov, Y. & Wellner, J. A. — Efficient and Adaptive Estimation for Semiparametric Models]}.

Synthesis. The foundational reason a single number $I({\theta }_0{)}^{-1}$ governs all of asymptotic estimation is that the local experiment converges to one Gaussian observation of the shift 45.04.06, and in that experiment the efficient estimator is shift-equivariant with constant risk, while the Anderson lemma certifies that any independent perturbation or worst-case displacement only spreads a centred Gaussian. The convolution and minimax theorems are dual readings of this single fact: the convolution statement factors every regular law through $N(0,I^{-1})$, and the minimax statement bounds every law's worst-case neighbourhood risk by the Gaussian shift's, so being efficient is exactly being the unconvolved Gaussian, with residual $M={\delta }_0$. This is the same score-and-information mechanism that pins the MLE in 45.04.03; here it generalises from one estimator to a bound over the entire regular and (via the minimax form) irregular classes, and putting these together the superefficiency of Hodges is revealed as a local-minimax illusion rather than a genuine improvement. The semiparametric layer is dual to the parametric one: where the parametric bound inverts the full information matrix, the semiparametric bound projects a gradient onto the tangent space and takes the squared norm, and the central insight is that an unknown nuisance enters only through the part of the score it spans — efficiency is attainable adaptively precisely when the efficient score is orthogonal to the nuisance tangent space, the bridge from the finite-dimensional Cramér-Rao floor of 45.01.05 to the infinite-dimensional information bound.

Full proof set Master

The convolution theorem is proved in the Key theorem; the asymptotic-representation transport, the minimax bound via Anderson's lemma, and the efficient-influence-function projection are recorded here.

Proposition 1 (Anderson's lemma). Let $Z\sim N(0,\Sigma )$ on ${\mathbb{R}}^k$ with $\Sigma \succ 0$ and let $C$ be symmetric ($C=-C$) and convex. Then $\mathbb{P}(Z+v\in C)\le \mathbb{P}(Z\in C)$ for all $v\in {\mathbb{R}}^k$.

Proof. The density $\varphi (z)\propto \exp (-\frac{1}{2}{z}^{\mathsf{T}}{\Sigma }^{-1}z)$ is symmetric and log-concave, hence quasi-concave: its superlevel sets $\{z:\varphi (z)\ge t\}$ are symmetric convex bodies. Write $f=1_C$ and, for the displaced density $\varphi (\cdot -v)$, use the Prékopa-Leindler inequality (or the equivalent Brunn-Minkowski peakedness statement): for the symmetric convex $C$ and the symmetric unimodal $\varphi$, the function $v\mapsto {\int }_C\varphi (z-v)dz$ is itself symmetric and quasi-concave in $v$, so it attains its maximum at $v=0$. Concretely, by symmetry ${\int }_C\varphi (z-v)dz={\int }_C\varphi (z+v)dz$, and by convexity of $C$ and the midpoint $\frac{1}{2}((z+v)+(z-v))=z$,

$${\int }_C\varphi (z)dz={\int }_C\varphi \left(\frac{(z+v)+(z-v)}{2}\right)dz\ge {\int }_C\sqrt{\varphi (z-v)\varphi (z+v)}dz\ge {\int }_C\varphi (z-v)dz,$$

the first inequality from log-concavity of $\varphi$ and the second from the AM-GM/symmetry pairing on $C$. Hence $\mathbb{P}(Z\in C)\ge \mathbb{P}(Z+v\in C)$. For a bowl-shaped loss, the layer-cake formula $\ell (x)={\int }_0^{\infty }1\{x\notin C_c\}dc$ over the symmetric convex sublevel complements gives $\mathbb{E}\ell (Z+v)={\int }_0^{\infty }(1-\mathbb{P}(Z+v\in C_c))dc\ge {\int }_0^{\infty }(1-\mathbb{P}(Z\in C_c))dc=\mathbb{E}\ell (Z)$. $\square$

Proposition 2 (minimax value of the Gaussian shift experiment). In $\{N(h,I^{-1}):h\in {\mathbb{R}}^k\}$, for bowl-shaped $\ell$, ${\inf }_T{\sup }_h{\mathbb{E}}_h\ell (T-h)=\mathbb{E}\ell (Z)$ with $Z\sim N(0,I^{-1})$, the infimum over all randomised estimators $T(X)$ based on $X\sim N(h,I^{-1})$.

Proof. The estimator $T(X)=X$ has $T-h\sim N(0,I^{-1})$ for every $h$, so ${\sup }_h{\mathbb{E}}_h\ell (T-h)=\mathbb{E}\ell (Z)$, giving the upper bound. For the lower bound, put the prior $h\sim N(0,c\mathrm{Id})$. The posterior of $h$ given $X$ is Gaussian, and as $c\to \infty$ it converges to $N(X,I^{-1})$. For any estimator $T$, the integrated (Bayes) risk is $\ge$ the Bayes risk of the optimal estimator, whose posterior risk at $X$ is ${\inf }_t\mathbb{E}[\ell (t-h)\mid X]$; by Proposition 1 applied to the posterior $N(X,I^{-1})$ this infimum is attained at $t=X$ (the posterior mean / symmetry centre) and equals $\mathbb{E}\ell (Z)$ in the limit $c\to \infty$. Since the minimax risk dominates every Bayes risk, ${\inf }_T{\sup }_h{\mathbb{E}}_h\ell (T-h)\ge {\lim }_{c\to \infty }(\text{Bayes risk})=\mathbb{E}\ell (Z)$. The bounds coincide. $\square$

Proposition 3 (local asymptotic minimax, via the representation theorem). For any estimator sequence ${\hat{\theta }}_n$ and bowl-shaped $\ell$, ${\sup }_I{\mathrm{liminf}}_n{\sup }_{h\in I}{\mathbb{E}}_{{\theta }_0+h/\sqrt{n}}\ell (\sqrt{n}({\hat{\theta }}_n-{\theta }_0-h/\sqrt{n}))\ge \mathbb{E}\ell (Z)$, $Z\sim N(0,I({\theta }_0{)}^{-1})$.

Proof. Fix a finite $I\subset {\mathbb{R}}^k$. By the asymptotic-representation theorem (Proposition 4 below) the localised sequence $T_n=\sqrt{n}({\hat{\theta }}_n-{\theta }_0)$ is matched in the limit by a randomised statistic $T$ in $\{N(h,I^{-1})\}$: for each $h\in I$, ${\mathrm{liminf}}_n{\mathbb{E}}_{{\theta }_0+h/\sqrt{n}}\ell (T_n-h)\ge {\mathbb{E}}_h\ell (T-h)$ (the loss is bowl-shaped, hence lower-semicontinuous on a suitable truncation, and the portmanteau inequality for $\mathrm{liminf}$ over open/closed sets applies after truncating $\ell$ at level $m$ and letting $m\to \infty$). Therefore

$$\underset{n}{\mathrm{liminf}}\underset{h\in I}{\sup }{\mathbb{E}}_{{\theta }_0+h/\sqrt{n}}\ell (T_n-h)\ge \underset{h\in I}{\sup }{\mathbb{E}}_h\ell (T-h).$$

Taking the supremum over finite $I$ on both sides and using ${\sup }_I{\sup }_{h\in I}={\sup }_{h\in {\mathbb{R}}^k}$,

$$\underset{I}{\sup }\underset{n}{\mathrm{liminf}}\underset{h\in I}{\sup }\mathbb{E}\ell (T_n-h)\ge \underset{h\in {\mathbb{R}}^k}{\sup }{\mathbb{E}}_h\ell (T-h)\ge \underset{T^{\prime }}{\inf }\underset{h}{\sup }{\mathbb{E}}_h\ell (T^{\prime }-h)=\mathbb{E}\ell (Z),$$

the last equality by Proposition 2. $\square$

Proposition 4 (asymptotic-representation theorem; statement and convolution corollary). Under LAN at ${\theta }_0$, every localised estimator sequence $T_n=\sqrt{n}({\hat{\theta }}_n-{\theta }_0)$ that converges in distribution under each $P_{{\theta }_0+h/\sqrt{n},n}$ has, as its $h$-indexed family of limits, the $h$-indexed law family of a single randomised statistic $T$ in $\{N(h,I^{-1})\}$. If moreover ${\hat{\theta }}_n$ is regular, $T$ is shift-equivariant, so $T=\Delta +(\text{independent residual})$ and the limit law is $N(0,I^{-1})\ast M$.

Proof. The local experiments ${\mathcal{E}}_{n,{\theta }_0}=\{P_{{\theta }_0+h/\sqrt{n},n}\}$ converge to ${\mathcal{G}}_I=\{N(h,I^{-1})\}$ in the Le Cam deficiency distance 45.04.06; Le Cam's theory provides, for each weakly convergent localised sequence $T_n$, a Markov kernel (randomisation) $K$ from the observation $X\sim N(h,I^{-1})$ of the limit experiment producing $T=K(X)$ with ${\mathcal{L}}_h(T)={\lim }_n{\mathcal{L}}_{{\theta }_0+h/\sqrt{n},n}(T_n)$ for all $h$; this is the matching property of convergence in deficiency, applied to the bounded-loss decision problem of estimating $h$. If ${\hat{\theta }}_n$ is regular then ${\mathcal{L}}_h(T-h)$ is independent of $h$, i.e. $T$ is shift-equivariant in the Gaussian shift experiment. A shift-equivariant estimator of the mean of $X\sim N(h,I^{-1})$ has the form $T=X+U$ with $U$ independent of $X$ (the maximal-invariant decomposition: equivariance forces the estimation error $T-h=(X-h)+U$ to have an $h$-free law with $X-h\sim N(0,I^{-1})\perp U$). Writing $\Delta =I(X)$ recovers $T-h=N(0,I^{-1})+U$, so $\mathcal{L}(T-h)=N(0,I^{-1})\ast \mathcal{L}(U)$, the convolution with $M=\mathcal{L}(U)$. $\square$

Proposition 5 (efficient influence function as a projection; uniqueness). Let $\psi$ be pathwise differentiable at $P_{{\theta }_0}$ with tangent space $\mathcal{T}\subseteq L_0^2(P_{{\theta }_0})$ closed and gradients $\{\tilde{\psi }:\mathbb{E}[\tilde{\psi }g]=\dot{\psi }(g)\forall g\in \mathcal{T}\}$. The efficient influence function ${\tilde{\psi }}_{\text{eff}}=\Pi (\tilde{\psi }\mid \mathcal{T})$ is the same for every gradient and is the unique gradient lying in $\mathcal{T}$; the efficiency bound is $\Vert {\tilde{\psi }}_{\text{eff}}{\Vert }_{L^2(P_{{\theta }_0})}^2$.

Proof. Two gradients ${\tilde{\psi }}_1,{\tilde{\psi }}_2$ satisfy $\mathbb{E}[({\tilde{\psi }}_1-{\tilde{\psi }}_2)g]=0$ for all $g\in \mathcal{T}$, so ${\tilde{\psi }}_1-{\tilde{\psi }}_2\in {\mathcal{T}}^{\perp }$; their projections onto $\mathcal{T}$ coincide, proving ${\tilde{\psi }}_{\text{eff}}$ is gradient-independent. The projection $\Pi (\tilde{\psi }\mid \mathcal{T})$ lies in $\mathcal{T}$ and for $g\in \mathcal{T}$, $\mathbb{E}[\Pi (\tilde{\psi })g]=\mathbb{E}[\tilde{\psi }g]=\dot{\psi }(g)$ (projection preserves inner products with $\mathcal{T}$), so it is itself a gradient, and it is the unique gradient in $\mathcal{T}$ because any two such differ by an element of $\mathcal{T}\cap {\mathcal{T}}^{\perp }=\{0\}$. For the bound: a regular estimator's limit is $N(0,V)\ast M$ with $V\ge \mathbb{E}[{\tilde{\psi }}_{\text{eff}}^2]$, since the efficient estimator is asymptotically linear with influence function ${\tilde{\psi }}_{\text{eff}}$ and any other gradient adds an orthogonal ${\mathcal{T}}^{\perp }$-component of non-negative variance; the minimal variance is the squared norm of the in-tangent-space projection. $\square$

Connections Master

Local asymptotic normality 45.04.06 is the foundation on which this entire unit rests: the convolution and minimax bounds are statements about the Gaussian shift experiment $\{N(h,I({\theta }_0{)}^{-1})\}$ that LAN produces as the local limit, and the central sequence ${\Delta }_{n,{\theta }_0}$ is the sufficient statistic in terms of which the efficient estimator ${\theta }_0+n^{-1/2}{I}^{-1}{\Delta }_{n,{\theta }_0}$ is written; without the convergence-of-experiments statement there would be no limit model in which to compute optimality.

Contiguity and Le Cam's third lemma 45.04.07 are the load-bearing tool of the convolution proof: regularity is a statement about the estimator's limit law under the contiguous local alternatives, and the third lemma's mean-shift-by-covariance is exactly what forces $\mathrm{Cov}(S,\Delta )=\mathrm{Id}$, from which the independent residual and the convolution factorisation follow; the same change-of-measure machinery transports the Gaussian-shift lower bounds back to the original sequence.

The asymptotic normality and efficiency of the MLE 45.04.03 is the estimator-level instance of the convolution theorem: the MLE is regular with residual $M={\delta }_0$, so its limit law $N(0,I^{-1})$ is the unconvolved efficient factor, and the superefficiency exhibited by Hodges' estimator there is precisely the regularity failure this unit isolates and the local asymptotic minimax theorem penalises with divergent neighbourhood risk.

The Fisher information and Cramér-Rao lower bound of 45.01.05 are recovered and upgraded: the finite-sample variance floor $I({\theta }_0{)}^{-1}$ reappears as the covariance of the efficient Gaussian factor and as the irreducible factor in every regular estimator's limit law, and the semiparametric efficiency bound generalises $I^{-1}$ to the squared norm of the efficient influence function, the projection of a gradient onto the tangent space, with the nuisance-known information $I_{\theta \theta }$ replaced by the smaller efficient information $\tilde{I}$.

The asymptotic relative efficiency of estimators and tests 45.04.05 is the comparative reading of the convolution theorem: the ratio of efficient asymptotic variances $I^{-1}$ across procedures is the ARE, the convolution theorem identifies the variance floor each procedure is measured against, and a procedure's ARE relative to the efficient one is $1$ iff its convolution residual $M$ is ${\delta }_0$, the same efficiency criterion in ratio form.

Historical & philosophical context Master

The optimality theory of asymptotic estimation was completed by Jaroslav Hájek in two papers. The 1970 convolution theorem ^{[Hájek, J. — A characterization of limiting distributions of regular estimates]} characterised the limit laws of regular estimators in a LAN family as exactly the convolutions $N(0,I^{-1})\ast M$, establishing that the efficient Gaussian law is a factor of every regular limit and so cannot be improved upon; the 1972 local asymptotic minimax theorem ^{[Hájek, J. — Local asymptotic minimax and admissibility in estimation]} removed the regularity hypothesis by bounding the worst-case risk over shrinking neighbourhoods, resolving the superefficiency paradox that Lucien Le Cam had exhibited in 1953 and that Joseph Hodges had captured in his shrinkage example. The two theorems made precise a notion of asymptotic efficiency that the older Fisher-Cramér-Rao programme could only approximate: not a lower bound on variance attained by a single estimator, but a structural bound on the entire class of reasonable estimators.

The framework rests on Le Cam's reconceptualisation of the experiment as the object of asymptotic study and his deficiency distance in which sequences of experiments converge ^{[Le Cam, L. — Asymptotic Methods in Statistical Decision Theory]}; the asymptotic-representation theorem that transports Gaussian-shift bounds back to the original problem is its central consequence. The semiparametric extension — replacing the finite-dimensional information by the efficient influence function, the projection of a gradient onto the model's tangent space — was systematised by Peter Bickel, Chris Klaassen, Yaacov Ritov, and Jon Wellner ^{[Bickel, P. J., Klaassen, C. A. J., Ritov, Y. & Wellner, J. A. — Efficient and Adaptive Estimation for Semiparametric Models]}, building on the convolution-theorem apparatus and on the Hilbert-space geometry of scores, and it underlies the modern theory of doubly-robust and orthogonal estimation. The analytic peakedness lemma making the centred Gaussian optimal for every symmetric convex loss is due to Theodore Anderson in 1955.

Bibliography Master

@article{Hajek1970,
  author  = {H\'ajek, Jaroslav},
  title   = {A characterization of limiting distributions of regular estimates},
  journal = {Zeitschrift f\"ur Wahrscheinlichkeitstheorie und Verwandte Gebiete},
  volume  = {14},
  year    = {1970},
  pages   = {323--330}
}

@inproceedings{Hajek1972,
  author    = {H\'ajek, Jaroslav},
  title     = {Local asymptotic minimax and admissibility in estimation},
  booktitle = {Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability},
  volume    = {1},
  publisher = {University of California Press},
  year      = {1972},
  pages     = {175--194}
}

@book{vanderVaart1998,
  author    = {van der Vaart, Aad W.},
  title     = {Asymptotic Statistics},
  publisher = {Cambridge University Press},
  series    = {Cambridge Series in Statistical and Probabilistic Mathematics},
  year      = {1998}
}

@book{BKRW1993,
  author    = {Bickel, Peter J. and Klaassen, Chris A. J. and Ritov, Ya'acov and Wellner, Jon A.},
  title     = {Efficient and Adaptive Estimation for Semiparametric Models},
  publisher = {Johns Hopkins University Press},
  year      = {1993}
}

@book{LeCam1986,
  author    = {Le Cam, Lucien},
  title     = {Asymptotic Methods in Statistical Decision Theory},
  series    = {Springer Series in Statistics},
  publisher = {Springer},
  year      = {1986}
}

@article{Anderson1955,
  author  = {Anderson, Theodore W.},
  title   = {The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities},
  journal = {Proceedings of the American Mathematical Society},
  volume  = {6},
  number  = {2},
  year    = {1955},
  pages   = {170--176}
}

@article{LeCam1953,
  author  = {Le Cam, Lucien},
  title   = {On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates},
  journal = {University of California Publications in Statistics},
  volume  = {1},
  year    = {1953},
  pages   = {277--330}
}