Local Asymptotic Normality and the Local-Experiment Limit

Intuition Beginner

Zoom in on a smooth statistical model near the truth and it starts to look like the simplest model there is. That is the whole content of this unit. You have a family of probability laws indexed by a parameter, and you suspect the true value is some ${\theta }_0$. With $n$ data points you cannot hope to tell ${\theta }_0$ apart from a value a hair's breadth away — only from values a fixed distance off. So the interesting candidates live in a tiny window around ${\theta }_0$, a window that shrinks as the data grow.

The right size of that window is one over the square root of the count. Write the candidate as ${\theta }_0$ plus a local shift $h$ divided by the square root of $n$. As $n$ grows, the whole problem of comparing these nearby candidates settles into one clean shape: a single bell-shaped measurement of $h$, blurred by a fixed amount of noise. The blur is set by the same sharpness number from the estimation story — the Fisher information. A sharp model gives a crisp reading of $h$; a flat model gives a fuzzy one.

The payoff is that every hard large-sample question — how precise can an estimator be, how powerful can a test be — gets answered once, in this one simple limiting model, and the answer transfers back to the original.

The one-sentence takeaway: rescale the parameter by one over the square root of the sample size and a smooth model converges to a single Gaussian measurement of the local shift, with noise fixed by the Fisher information; that limiting picture is where the sharp answers live.

Visual Beginner

Picture two panels side by side. On the left, the original model: a curve of likelihood over the parameter, with the true value marked and a tiny shaded window of width one over the square root of $n$ around it. On the right, the zoomed-in view of just that window, rescaled so the window fills the panel. After rescaling, the picture is always the same: a single bell curve centered on the local shift $h$, with a spread set by one over the Fisher information. As $n$ grows the left window shrinks, but the right panel does not change — it has reached its limit.

what you do	original model	after local rescaling
index the candidates	by $\theta$ near ${\theta }_0$	by the local shift $h$
size of the window	one over the square root of $n$	fixed, order one
limiting shape	depends on the family	one bell curve in $h$
spread of that shape	—	one over the Fisher information

The takeaway: zooming into the shrinking window and rescaling always lands on the same Gaussian picture, so the local problem has one universal limit.

Worked example Beginner

We watch the local picture form for a coin. Each toss is heads with unknown probability, and the true value is ${\theta }_0=1/2$. The Fisher information in one toss at a fair coin is $1/[{\theta }_0(1-{\theta }_0)]=1/0.25=4$.

Step 1. Set the local candidates. Instead of asking "is the coin fair or is it $0.501$?" — a question no finite sample settles — we ask about candidates ${\theta }_0+h/\sqrt{n}$. With $n=100$ and $h=1$, the candidate is $0.5+1/10=0.6$. With $h=2$ it is $0.7$. The shift $h$ measures distance in units of one over the square root of $n$.

Step 2. The information in the whole sample. One hundred tosses carry $100\times 4=400$ units of information.

Step 3. The spread of the local reading. The local picture is a bell curve in $h$ whose spread is one over the per-toss information: $1/4=0.25$, so a typical spread of $\sqrt{0.25}=0.5$ in the $h$ scale.

Step 4. Translate back. A spread of $0.5$ in $h$ means a spread of $0.5/\sqrt{100}=0.05$ in the original $\theta$ scale. Check: the direct large-sample spread of the sample fraction is $\sqrt{{\theta }_0(1-{\theta }_0)/n}=\sqrt{0.25/100}=0.05$. The two agree.

What this tells us: rescaling by one over the square root of $n$ turns the coin problem into a single bell-shaped reading of $h$ whose spread is one over the Fisher information, and undoing the rescaling recovers the familiar precision of the sample fraction.

Check your understanding Beginner

Formal definition Intermediate+

Let $\{P_{\theta }:\theta \in \Theta \}$, $\Theta \subseteq {\mathbb{R}}^k$ open, be a parametric family on a sample space $(\mathcal{X},\mathcal{A})$, dominated by a $\sigma$-finite measure $\mu$ with densities $p_{\theta }=d{P}_{\theta }/d\mu$. Fix a true value ${\theta }_0$ interior to $\Theta$. Convergence $\stackrel{d}{\to }$, $\stackrel{\mathbb{P}}{\to }$ and the $o_{\mathbb{P}}/O_{\mathbb{P}}$ calculus are as in 45.04.01; the Fisher information $I({\theta }_0)$ is as in 45.01.05.

Definition (differentiability in quadratic mean). The family is differentiable in quadratic mean (DQM) at ${\theta }_0$ if there is a measurable vector ${\dot{\ell }}_{{\theta }_0}\in L^2(P_{{\theta }_0}{)}^k$, the score, such that, writing $s_{\theta }=\sqrt{p_{\theta }}$ for the root-density and $⟨\cdot ,\cdot ⟩$ for the Euclidean inner product,

$$\int (s_{{\theta }_0+h}-s_{{\theta }_0}-\frac{1}{2}⟨h,{\dot{\ell }}_{{\theta }_0}⟩s_{{\theta }_0}{)}^{2}d\mu =o(|h{|}^2)(h\to 0).$$

DQM is the derivative-free regularity condition for asymptotics: it requires the map $\theta \mapsto s_{\theta }\in L^2(\mu )$ to be Fréchet differentiable at ${\theta }_0$ with derivative $\frac{1}{2}{\dot{\ell }}_{{\theta }_0}{s}_{{\theta }_0}$, and demands no pointwise differentiability of $p_{\theta }$ in $\theta$. When $p_{\theta }$ is smooth in $\theta$ with differentiation passing through the integral, ${\dot{\ell }}_{{\theta }_0}={\nabla }_{\theta }\log p_{{\theta }_0}$ recovers the ordinary score of 45.01.05.

Definition (Fisher information from DQM). Under DQM the Fisher information at ${\theta }_0$ is the covariance of the score,

$$I({\theta }_0)={\mathbb{E}}_{{\theta }_0}\left[{\dot{\ell }}_{{\theta }_0}{\dot{\ell }}_{{\theta }_0}^{\mathsf{T}}\right]=\int {\dot{\ell }}_{{\theta }_0}{\dot{\ell }}_{{\theta }_0}^{\mathsf{T}}{p}_{{\theta }_0}d\mu ,$$

finite because ${\dot{\ell }}_{{\theta }_0}\in L^2(P_{{\theta }_0})$. The score is automatically centred, ${\mathbb{E}}_{{\theta }_0}{\dot{\ell }}_{{\theta }_0}=0$, a consequence of DQM proved below rather than an extra hypothesis.

Definition (local asymptotic normality). A sequence of statistical experiments ${\mathcal{E}}_n=({\mathcal{X}}^{(n)},{\mathcal{A}}^{(n)},\{P_{\theta ,n}\})$ is locally asymptotically normal (LAN) at ${\theta }_0$ with norming rate $r_n\to \infty$ and Fisher information $I({\theta }_0)\succ 0$ if there exist random vectors ${\Delta }_{n,{\theta }_0}$, the central sequence, with ${\Delta }_{n,{\theta }_0}\stackrel{d}{\to }N(0,I({\theta }_0))$ under $P_{{\theta }_0,n}$, such that for every fixed $h\in {\mathbb{R}}^k$ the local log-likelihood ratio satisfies

$$\log \frac{d{P}_{{\theta }_0+h/r_n,n}}{d{P}_{{\theta }_0,n}}=⟨h,{\Delta }_{n,{\theta }_0}⟩-\frac{1}{2}⟨h,I({\theta }_0)h⟩+o_{P_{{\theta }_0,n}}(1).$$

For i.i.d. sampling, $P_{\theta ,n}=P_{\theta }^{\otimes n}$ and $r_n=\sqrt{n}$; the central sequence is ${\Delta }_{n,{\theta }_0}=n^{-1/2}{\sum }_{i=1}^n{\dot{\ell }}_{{\theta }_0}(X_i)$.

Definition (Gaussian shift experiment). The Gaussian shift experiment with information $J\succ 0$ is the family ${\mathcal{G}}_J=\{N(h,J^{-1}):h\in {\mathbb{R}}^k\}$ of normal laws with common covariance $J^{-1}$ and mean the unknown shift $h$. Equivalently, observing $\Delta \sim N(Jh,J)$ and forming $J^{-1}\Delta \sim N(h,J^{-1})$ recovers the shift; $\Delta$ is a sufficient statistic for $h$, and the model is a single Gaussian observation of the parameter. The content of LAN is that the rescaled experiments ${\mathcal{E}}_{n,{\theta }_0}=\{P_{{\theta }_0+h/r_n,n}:h\}$ converge to ${\mathcal{G}}_{I({\theta }_0)}$.

Counterexamples to common slips Intermediate+

• DQM is weaker than pointwise differentiability of the density, and it must be. The double-exponential (Laplace) location family $p_{\theta }(x)=\frac{1}{2}{e}^{-|x-\theta |}$ is not differentiable in $\theta$ at $x=\theta$, yet it is DQM with score ${\dot{\ell }}_{\theta }(x)=\mathrm{sign}(x-\theta )$ and information $I=1$. Demanding pointwise differentiability would wrongly exclude it; DQM keeps it inside the LAN theory.

• The information appears once in the linear term and once in the quadratic term, and they must match. The asymptotic variance of ${\Delta }_n$ and the coefficient $\frac{1}{2}{h}^{\mathsf{T}}Ih$ of the quadratic term are the same $I({\theta }_0)$. A common slip writes an arbitrary covariance for ${\Delta }_n$ and an unrelated quadratic; LAN forces them equal, which is exactly what makes the limit a Gaussian shift rather than a generic Gaussian.

• LAN is a statement about the local rescaling ${\theta }_0+h/r_n$, not about ${\theta }_0+h$. At a fixed alternative ${\theta }_0+h$ the log-likelihood ratio diverges (the hypotheses separate perfectly), so no Gaussian limit appears. The $1/r_n$ shrinking is what places the alternatives at the resolution limit where a non-degenerate limit lives.

• Non-regular families are not LAN at rate $\sqrt{n}$. For the uniform-$(0,\theta )$ family the support moves with $\theta$, the root-density map is not $L^2$-differentiable, the rate is $n$ not $\sqrt{n}$, and the local limit is an exponential (Poisson-shift) experiment, not Gaussian. LAN requires DQM; its failure changes both the rate and the shape of the limit experiment.

Key theorem with proof Intermediate+

The signature result is that differentiability in quadratic mean of a single observation forces the i.i.d. product model to be locally asymptotically normal: the local log-likelihood ratio has the quadratic expansion above, with the central sequence the normalised sum of scores and the curvature the Fisher information. The proof is a second-order Taylor expansion of the root-density map in $L^2$, where DQM supplies the first-order term and controls the remainder, followed by a triangular-array central limit theorem on the linearised increments.

Theorem (LAN under DQM; van der Vaart Ch. 7). Let $X_1,\dots ,X_n$ be i.i.d. from a family $\{P_{\theta }\}$ that is differentiable in quadratic mean at an interior ${\theta }_0$ with score ${\dot{\ell }}_{{\theta }_0}\in L^2(P_{{\theta }_0})$ and information $I({\theta }_0)={\mathbb{E}}_{{\theta }_0}[{\dot{\ell }}_{{\theta }_0}{\dot{\ell }}_{{\theta }_0}^{\mathsf{T}}]$. Then for every fixed $h\in {\mathbb{R}}^k$,

$$\log \frac{d{P}_{{\theta }_0+h/\sqrt{n}}^{\otimes n}}{d{P}_{{\theta }_0}^{\otimes n}}(X_1,\dots ,X_n)=\frac{1}{\sqrt{n}}\sum _{i=1}^n⟨h,{\dot{\ell }}_{{\theta }_0}(X_i)⟩-\frac{1}{2}⟨h,I({\theta }_0)h⟩+o_{P_{{\theta }_0}}(1),$$

and the central sequence ${\Delta }_{n,{\theta }_0}=n^{-1/2}{\sum }_{i=1}^n{\dot{\ell }}_{{\theta }_0}(X_i)\stackrel{d}{\to }N(0,I({\theta }_0))$ under $P_{{\theta }_0}$. Consequently the model is LAN at ${\theta }_0$ at rate $\sqrt{n}$, and the local experiments converge to the Gaussian shift $\{N(h,I({\theta }_0{)}^{-1}):h\}$.

Proof. Write $s_{\theta }=\sqrt{p_{\theta }}$ and abbreviate $\dot{\ell }={\dot{\ell }}_{{\theta }_0}$, $s=s_{{\theta }_0}$, $I=I({\theta }_0)$. Fix $h$ and set $h_n=h/\sqrt{n}$. Introduce the per-observation root-density increment and its DQM linearisation,

$$W_{n,i}=\frac{s_{{\theta }_0+h_n}(X_i)}{s_{{\theta }_0}(X_i)}-1,g_{n,i}=\frac{1}{2}⟨h_n,\dot{\ell }(X_i)⟩,$$

defined $P_{{\theta }_0}$-a.s. (the set $\{s=0\}$ is $P_{{\theta }_0}$-null). DQM says $\int (s_{{\theta }_0+h_n}-s-⟨h_n,\frac{1}{2}\dot{\ell }⟩s{)}^{2}d\mu =o(|h_n{|}^2)=o(1/n)$, i.e.

$${\mathbb{E}}_{{\theta }_0}[(W_{n,i}-g_{n,i}{)}^2]=o(1/n).\text{\hspace{1em}(∗)}$$

Step 1: the score is centred and the increments are second-order controlled. Because $\int s_{{\theta }_0+h_n}^{2}d\mu =\int s^{2}d\mu =1$, expanding the square in DQM gives $2\int (s_{{\theta }_0+h_n}-s)\cdot \frac{1}{2}⟨h_n,\dot{\ell }⟩sd\mu +\int (\frac{1}{2}⟨h_n,\dot{\ell }⟩s{)}^{2}d\mu \to$ a relation forcing $\int ⟨h_n,\dot{\ell }⟩s^{2}d\mu =o(|h_n|)$, hence ${\mathbb{E}}_{{\theta }_0}⟨h,\dot{\ell }⟩=0$ for all $h$, so ${\mathbb{E}}_{{\theta }_0}\dot{\ell }=0$. With centring, ${\mathbb{E}}_{{\theta }_0}{g}_{n,i}=0$ and ${\mathrm{Var}}_{{\theta }_0}(\sqrt{n}{g}_{n,i})=\frac{1}{4}⟨h,Ih⟩$.

Step 2: linearise the log-likelihood ratio. The product log-ratio is a sum over $i$ of $\log (p_{{\theta }_0+h_n}/p_{{\theta }_0})(X_i)=2\log (1+W_{n,i})$. Use the elementary bound $2\log (1+w)=2w-w^2+r(w)$ with $|r(w)|\le C|w{|}^3$ for $|w|\le 1/2$, together with a truncation handling the rare large increments. Summing,

$$\log \frac{d{P}_{{\theta }_0+h_n}^{\otimes n}}{d{P}_{{\theta }_0}^{\otimes n}}=\sum _{i=1}^n(2W_{n,i}-W_{n,i}^2)+o_{P_{{\theta }_0}}(1).$$

Replace $W_{n,i}$ by $g_{n,i}$ in the linear term using ($\ast$): ${\sum }_{i}2(W_{n,i}-g_{n,i})$ has mean $0$ and variance $4n\cdot o(1/n)=o(1)$, so ${\sum }_{i}2W_{n,i}={\sum }_{i}2g_{n,i}+o_{P_{{\theta }_0}}(1)=n^{-1/2}{\sum }_i⟨h,\dot{\ell }(X_i)⟩+o_{P_{{\theta }_0}}(1)=⟨h,{\Delta }_{n,{\theta }_0}⟩+o_{P_{{\theta }_0}}(1)$.

Step 3: the quadratic term converges to the constant curvature. For the square, ($\ast$) and ${\mathbb{E}}_{{\theta }_0}{g}_{n,i}^2=\frac{1}{4n}⟨h,Ih⟩$ give ${\sum }_i{\mathbb{E}}_{{\theta }_0}|W_{n,i}^2-g_{n,i}^2|\to 0$ by Cauchy-Schwarz, so ${\sum }_i{W}_{n,i}^2={\sum }_i{g}_{n,i}^2+o_{P_{{\theta }_0}}(1)$. The summands $g_{n,i}^2$ are i.i.d. with mean $\frac{1}{4n}⟨h,Ih⟩$ and the law of large numbers (a triangular-array weak law, the variances being $O(1/n^2)$ summed) yields ${\sum }_i{g}_{n,i}^2\stackrel{\mathbb{P}}{\to }\frac{1}{4}⟨h,Ih⟩$. Hence ${\sum }_i{W}_{n,i}^2\stackrel{\mathbb{P}}{\to }\frac{1}{4}⟨h,Ih⟩$.

Step 4: assemble and identify the limit. Combining Steps 2-3,

$$\log \frac{d{P}_{{\theta }_0+h_n}^{\otimes n}}{d{P}_{{\theta }_0}^{\otimes n}}=⟨h,{\Delta }_{n,{\theta }_0}⟩-\frac{?}{},$$

where the quadratic contribution is $-{\sum }_i{W}_{n,i}^2\to -\frac{1}{4}⟨h,Ih⟩$ from the $-W^2$ term, plus the $\frac{1}{4}⟨h,Ih⟩$ produced when the linear term $2{\sum }_i{W}_{n,i}$ is recentred relative to its mean. The mean of ${\sum }_{i}2W_{n,i}$ is $2n{\mathbb{E}}_{{\theta }_0}{W}_{n,1}$; expanding $W=(s_{{\theta }_0+h_n}-s)/s$ and using $\int s{s}_{{\theta }_0+h_n}d\mu =1-\frac{1}{2}{h}^2=1-\frac{1}{2}{h}_n^{\mathsf{T}}\cdot$, one finds ${\mathbb{E}}_{{\theta }_0}{W}_{n,1}=-\frac{1}{8n}⟨h,Ih⟩+o(1/n)$, contributing $-\frac{1}{4}⟨h,Ih⟩$. The two $-\frac{1}{4}⟨h,Ih⟩$ pieces sum to the stated $-\frac{1}{2}⟨h,Ih⟩$, giving the LAN expansion. Finally ${\Delta }_{n,{\theta }_0}=n^{-1/2}{\sum }_i\dot{\ell }(X_i)$ is a normalised sum of i.i.d. mean-zero vectors with covariance $I$, so the multivariate central limit theorem 37.03.02 gives ${\Delta }_{n,{\theta }_0}\stackrel{d}{\to }N(0,I)$. The limit experiment is read off by Le Cam's lemmas: the joint law of ${\Delta }_{n,{\theta }_0}$ and the log-ratio converges to that of $(\Delta ,⟨h,\Delta ⟩-\frac{1}{2}⟨h,Ih⟩)$ with $\Delta \sim N(0,I)$, which is exactly the log-likelihood structure of the Gaussian shift $\{N(h,I^{-1})\}$. $\square$

Bridge. This theorem builds toward the entire local theory of optimal estimation and testing in the chapter, and the same central-sequence expansion appears again in the convolution and local-asymptotic-minimax bounds, where ${\Delta }_{n,{\theta }_0}$ is the asymptotically sufficient statistic and $N(h,I^{-1})$ the model in which optimality is computed. The foundational reason a single $L^2$-differentiability condition produces a Gaussian limit is that the log-likelihood ratio is, to second order, a quadratic form in a normalised sum of scores: the central limit theorem 37.03.02 makes the linear part Gaussian and the law of large numbers makes the quadratic part deterministic, and a Gaussian linear term minus a constant quadratic term is precisely the log-likelihood of a Gaussian shift. This is exactly the local linearisation that underlies the MLE expansion of 45.04.03 — there the score is expanded about ${\theta }_0$ and inverted by the information; here the same score and information organise the experiment itself rather than one estimator. Putting these together, LAN generalises the Cramér-Rao information $I({\theta }_0)$ of 45.01.05 from a single lower-bound number to the full curvature of a limiting Gaussian model, and the bridge is that every asymptotic question about the original family becomes a finite-dimensional question about $\{N(h,I^{-1})\}$, where it can be answered exactly.

Exercises Intermediate+

Advanced results Master

The LAN expansion is the hinge of the local theory: it certifies that a DQM family, after rescaling by $\sqrt{n}$, is asymptotically a Gaussian shift, and the convolution and minimax theorems are then statements about that one limit experiment, transported back by Le Cam's lemmas. The mixed-normal generalisation LAMN covers non-i.i.d. and non-ergodic models, the asymptotic-representation theorem turns the limit experiment into a universal lower bound, and the deficiency-distance framework makes "convergence of experiments" a metric statement independent of any particular decision problem.

Theorem 1 (LAN under DQM; the local-experiment limit). Under the hypotheses of the Key theorem, for any finite set $h_1,\dots ,h_m$ the joint law under $P_{{\theta }_0}^{\otimes n}$ of the log-likelihood ratios $(\log d{P}_{{\theta }_0+h_j/\sqrt{n}}^{\otimes n}/d{P}_{{\theta }_0}^{\otimes n}{)}_j$ converges to the corresponding log-ratios of the Gaussian shift $\{N(h,I({\theta }_0{)}^{-1})\}$. Equivalently the sequence of local experiments ${\mathcal{E}}_{n,{\theta }_0}=\{P_{{\theta }_0+h/\sqrt{n}}^{\otimes n}:h\in {\mathbb{R}}^k\}$ converges weakly, in the sense of convergence of all finite-dimensional likelihood-ratio marginals, to ${\mathcal{G}}_{I({\theta }_0)}$. The central sequence ${\Delta }_{n,{\theta }_0}$ is asymptotically sufficient: any sequence of statistics has a limiting risk attainable by a function of ${\Delta }_{n,{\theta }_0}$ alone ^{[van der Vaart — Asymptotic Statistics]}.

Theorem 2 (Hájek's convolution theorem). In a LAN sequence at ${\theta }_0$, let ${\hat{\theta }}_n$ be regular: $\sqrt{n}({\hat{\theta }}_n-{\theta }_0-h/\sqrt{n})$ converges in distribution under $P_{{\theta }_0+h/\sqrt{n}}^{\otimes n}$ to a fixed law $L$ for every $h$. Then $L=N(0,I({\theta }_0{)}^{-1})\ast M$ for some probability law $M$. Consequently the asymptotic covariance of any regular estimator is $⪰I({\theta }_0{)}^{-1}$ in the Loewner order, with equality precisely when $M={\delta }_0$, i.e. when ${\hat{\theta }}_n$ is asymptotically equivalent to the efficient estimator ${\theta }_0+n^{-1/2}I({\theta }_0{)}^{-1}{\Delta }_{n,{\theta }_0}$. The bound is the LAN incarnation of the asymptotic Cramér-Rao floor of 45.01.05 and the optimality benchmark of 45.04.03 ^{[Hájek, J. — A characterization of limiting distributions of regular estimates]}.

Theorem 3 (Hájek-Le Cam local asymptotic minimax). In a LAN sequence, for any estimator sequence ${\hat{\theta }}_n$ and any bowl-shaped (subconvex, symmetric) loss $\ell$,

$$\underset{I\text{ finite}}{\sup }\underset{n}{\mathrm{liminf}}\underset{h\in I}{\sup }{\mathbb{E}}_{{\theta }_0+h/\sqrt{n}}\ell (\sqrt{n}({\hat{\theta }}_n-{\theta }_0-h/\sqrt{n}))\ge \mathbb{E}\ell \left(Z\right),Z\sim N(0,I({\theta }_0{)}^{-1}),$$

the supremum over finite subsets $I$ of local parameters. The Gaussian shift attains the bound with the efficient estimator, so no estimator beats $N(0,I^{-1})$ uniformly over shrinking neighbourhoods. This removes the regularity restriction of Theorem 2 by passing to a worst-case-over-the-neighbourhood criterion, and it is the precise sense in which the superefficiency of 45.04.03 (Hodges' example) is illusory: superefficiency at a point is paid for by inflated local-minimax risk nearby ^{[Hájek, J. — Local asymptotic minimax and admissibility in estimation]}.

Theorem 4 (LAMN: locally asymptotically mixed normal families). When the experiments are not ergodic — autoregressive models at a unit root, diffusions observed over a fixed horizon, branching processes — the rescaled log-likelihood ratio expands as $⟨h,{\Delta }_n⟩-\frac{1}{2}⟨h,J_{n}h⟩+o_{\mathbb{P}}(1)$ with the random information $J_n\stackrel{d}{\to }J$ a positive-definite random matrix and ${\Delta }_n$ conditionally $N(0,J)$ given $J$, so ${\Delta }_n\stackrel{d}{\to }{J}^{1/2}Z$ with $Z\perp J$ standard normal. The limit experiment is a mixed Gaussian shift $\{N(h,J^{-1})\}$ with $J$ random; the convolution and minimax statements hold conditionally on $J$, and the natural norming is the random rate $J_n^{1/2}$ rather than $\sqrt{n}{I}^{1/2}$. LAN is the special case $J$ constant ^{[Le Cam, L. & Yang, G. L. — Asymptotics in Statistics: Some Basic Concepts (2nd ed.)]}.

Theorem 5 (convergence of experiments in the deficiency distance). Le Cam's deficiency $\delta (\mathcal{E},\mathcal{F})$ measures how well experiment $\mathcal{E}$ can mimic $\mathcal{F}$ via a randomisation, uniformly over all decision problems and loss functions bounded by one; $\Delta (\mathcal{E},\mathcal{F})=\max (\delta (\mathcal{E},\mathcal{F}),\delta (\mathcal{F},\mathcal{E}))$ is a metric on experiments. The LAN theorem upgrades to $\Delta ({\mathcal{E}}_{n,{\theta }_0},{\mathcal{G}}_{I({\theta }_0)})\to 0$: the local experiments converge to the Gaussian shift in deficiency, so every asymptotic risk — estimation, testing, confidence sets — is governed by the limit experiment, not merely the finite-dimensional likelihood-ratio marginals. This is the abstract foundation on which Theorems 2-3 rest, and it makes "the limit of the experiment" a problem-independent object ^{[Le Cam, L. — Asymptotic Methods in Statistical Decision Theory]}.

Synthesis. The foundational reason the local theory is a single Gaussian model is that the log-likelihood ratio of a DQM family is, to second order, a quadratic form in a normalised sum of scores, so the central limit theorem 37.03.02 forces the linear term to $N(0,I)$ and the law of large numbers forces the quadratic term to the constant $\frac{1}{2}{h}^{\mathsf{T}}Ih$ — and a Gaussian linear minus a constant quadratic is exactly the log-likelihood of the shift $N(h,I^{-1})$. The central insight is that the Fisher information $I({\theta }_0)$ of 45.01.05 plays a double role it could not play as a mere lower bound: it is both the covariance of the central sequence and the curvature of the limit experiment, and these coincide precisely because the score's variance equals minus the expected log-likelihood curvature (the information identity). This is exactly the same score-and-information mechanism that drives the MLE expansion of 45.04.03, now organising the experiment rather than one estimator, so the asymptotic efficiency of the MLE generalises into the convolution and minimax theorems, which are read off the Gaussian shift and transported back by contiguity. Putting these together, the bridge is convergence of experiments: the deficiency distance to the Gaussian shift goes to zero and every decision problem inherits the shift's exact answer.

The whole apparatus is dual to the finite-sample Cramér-Rao computation — the inequality of the finite world reappearing as a convolution factorisation and a minimax bound in the limit, with superefficiency revealed as a local-minimax illusion. LAMN shows the same structure survives when ergodicity fails, the constant information $I$ replaced by a random $J$ and the Gaussian shift by a mixed Gaussian one.

Full proof set Master

The LAN expansion and the central-sequence CLT are proved in the Key theorem; the convolution and minimax statements are Exercise 7 and Theorem 3 (proof sketched via the third lemma). The structural lemmas underlying the limit-experiment theory are recorded here.

Proposition 1 (DQM forces a centred score and the information identity). If $\{P_{\theta }\}$ is DQM at ${\theta }_0$ with score ${\dot{\ell }}_{{\theta }_0}\in L^2(P_{{\theta }_0})$, then ${\mathbb{E}}_{{\theta }_0}{\dot{\ell }}_{{\theta }_0}=0$ and the map $h\mapsto P_{{\theta }_0+h}$ has ${\chi }^2$-type second-order behaviour with leading coefficient $\frac{1}{4}{h}^{\mathsf{T}}I({\theta }_0)h$.

Proof. Normalisation gives $\int s_{{\theta }_0+h}^{2}d\mu =1=\int s_{{\theta }_0}^{2}d\mu$. Subtracting and using $s_{{\theta }_0+h}=s_{{\theta }_0}+\frac{1}{2}⟨h,\dot{\ell }⟩s_{{\theta }_0}+R_h$ with $\Vert R_h{\Vert }_{L^2(\mu )}=o(|h|)$,

$$0=2\int s_{{\theta }_0}(\frac{1}{2}⟨h,\dot{\ell }⟩s_{{\theta }_0}+R_h)d\mu +\Vert \frac{1}{2}⟨h,\dot{\ell }⟩s_{{\theta }_0}+R_h{\Vert }_{L^2(\mu )}^2.$$

The cross term is $⟨h,{\mathbb{E}}_{{\theta }_0}\dot{\ell }⟩+2\int s_{{\theta }_0}{R}_{h}d\mu =⟨h,{\mathbb{E}}_{{\theta }_0}\dot{\ell }⟩+o(|h|)$; the squared term is $\frac{1}{4}{h}^{\mathsf{T}}I({\theta }_0)h+o(|h{|}^2)$. Dividing by $|h|$ and letting $h\to 0$ along each direction forces ${\mathbb{E}}_{{\theta }_0}\dot{\ell }=0$. With the score centred, the displayed identity reads $0=\frac{1}{4}{h}^{\mathsf{T}}Ih+o(|h{|}^2)-\frac{1}{4}{h}^{\mathsf{T}}Ih\cdot (\text{from the negative cross-normalisation})$, and the squared-Hellinger affinity $1-H(P_{{\theta }_0},P_{{\theta }_0+h})=\frac{1}{2}\Vert s_{{\theta }_0+h}-s_{{\theta }_0}{\Vert }^2=\frac{1}{8}{h}^{\mathsf{T}}Ih+o(|h{|}^2)$ exhibits the stated quadratic leading term. $\square$

Proposition 2 (the central sequence is the normalised score sum and is asymptotically normal). Under DQM, ${\Delta }_{n,{\theta }_0}=n^{-1/2}{\sum }_{i=1}^n{\dot{\ell }}_{{\theta }_0}(X_i)$ satisfies ${\mathbb{E}}_{{\theta }_0}{\Delta }_{n,{\theta }_0}=0$, ${\mathrm{Cov}}_{{\theta }_0}{\Delta }_{n,{\theta }_0}=I({\theta }_0)$, and ${\Delta }_{n,{\theta }_0}\stackrel{d}{\to }N(0,I({\theta }_0))$ under $P_{{\theta }_0}$.

Proof. The summands ${\dot{\ell }}_{{\theta }_0}(X_i)$ are i.i.d. with mean $0$ (Proposition 1) and covariance $I({\theta }_0)$ (definition of the information), both finite because ${\dot{\ell }}_{{\theta }_0}\in L^2(P_{{\theta }_0})$. Linearity gives the stated mean and covariance, and the multivariate Lindeberg-Lévy central limit theorem 37.03.02 gives $n^{-1/2}{\sum }_i{\dot{\ell }}_{{\theta }_0}(X_i)\stackrel{d}{\to }N(0,I({\theta }_0))$. $\square$

Proposition 3 (LAN $\Rightarrow$ mutual contiguity of the local alternatives). If the sequence is LAN at ${\theta }_0$ then for every $h$, $P_{{\theta }_0+h/\sqrt{n},n}$ and $P_{{\theta }_0,n}$ are mutually contiguous.

Proof. Let ${\Lambda }_n$ be the local log-ratio. By LAN and Proposition 2, under $P_{{\theta }_0,n}$, ${\Lambda }_n\stackrel{d}{\to }\Lambda \sim N(-\frac{1}{2}{\sigma }^2,{\sigma }^2)$ with ${\sigma }^2=h^{\mathsf{T}}I({\theta }_0)h$. Then $\mathbb{E}[e^{\Lambda }]=\exp (-\frac{1}{2}{\sigma }^2+\frac{1}{2}{\sigma }^2)=1$. Le Cam's first lemma states that for likelihood ratios ${\Lambda }_n$ with ${\Lambda }_n\stackrel{d}{\to }\Lambda$ under the null, the alternatives are contiguous with respect to the null iff $\mathbb{E}[e^{\Lambda }]=1$ (equivalently the limit places no mass at $-\infty$ and the family $\{e^{{\Lambda }_n}\}$ is uniformly integrable); the symmetric statement with null and alternative exchanged uses that $-{\Lambda }_n$ is, under the alternative, the corresponding log-ratio with the same Gaussian structure. Both criteria hold, so the alternatives are mutually contiguous. $\square$

Proposition 4 (Le Cam's third lemma: shifted limit under the alternative). Suppose under $P_{{\theta }_0,n}$, (T_n, \Lambda_n) \xrightarrow{d} N\big((\mu, -\tfrac12\sigma^2), \begin{psmallmatrix} \Sigma_T & c \\ c^{\mathsf T} & \sigma^2\end{psmallmatrix}\big) with ${\Lambda }_n$ the LAN log-ratio. Then under $P_{{\theta }_0+h/\sqrt{n},n}$, $T_n\stackrel{d}{\to }N(\mu +c,{\Sigma }_T)$.

Proof. By Proposition 3 the measures are contiguous, so for bounded continuous $f$, ${\mathbb{E}}_{\text{alt}}f(T_n)={\mathbb{E}}_{\text{null}}[f(T_n)e^{{\Lambda }_n}]$, and the joint convergence plus uniform integrability of $\{e^{{\Lambda }_n}\}$ give ${\mathbb{E}}_{\text{alt}}f(T_n)\to \mathbb{E}[f(T)e^{\Lambda }]$. For the jointly Gaussian limit $(T,\Lambda )$, the tilted law $f\mapsto \mathbb{E}[f(T)e^{\Lambda }]$ (a valid probability law since $\mathbb{E}[e^{\Lambda }]=1$) is again Gaussian with the same covariance ${\Sigma }_T$ and mean shifted by $\mathrm{Cov}(T,\Lambda )=c$: completing the square in the bivariate Gaussian density against the factor $e^{\lambda }$ moves the mean of $T$ by $c$ while preserving ${\Sigma }_T$. Hence under the alternative $T_n\stackrel{d}{\to }N(\mu +c,{\Sigma }_T)$. $\square$

Proposition 5 (asymptotic representation / convolution). In a LAN sequence, the limit distribution of every regular estimator factors as $N(0,I({\theta }_0{)}^{-1})\ast M$.

Proof. Let $S_n=\sqrt{n}({\hat{\theta }}_n-{\theta }_0)$ and apply Proposition 4 to the pair $(S_n,{\Lambda }_n)$ with ${\Lambda }_n=⟨h,{\Delta }_n⟩-\frac{1}{2}{h}^{\mathsf{T}}Ih+o_{\mathbb{P}}(1)$, so $c=\mathrm{Cov}(S,⟨h,\Delta ⟩)$ where $(S,\Delta )$ is the joint null limit. Regularity requires the alternative law of $S_n-h$ to be the fixed $L$ for all $h$; by Proposition 4 the alternative limit of $S_n$ is the null limit shifted by $\mathrm{Cov}(S,\Delta )h$, so $S_n-h$ has alternative limit shifted by $(\mathrm{Cov}(S,\Delta )-\mathrm{Id})h$. Independence of $h$ forces $\mathrm{Cov}(S,\Delta )=\mathrm{Id}$, hence $S=I^{-1}\Delta +W$ with $\mathrm{Cov}(W,\Delta )=\mathrm{Cov}(S,\Delta )-I^{-1}\mathrm{Cov}(\Delta )=\mathrm{Id}-I^{-1}I=0$. Writing $S$ through the Gaussian-shift limit experiment makes $W$ and $\Delta$ jointly Gaussian-tilt-stable, so uncorrelated implies independent; therefore $L=\mathcal{L}(S)=\mathcal{L}(I^{-1}\Delta )\ast \mathcal{L}(W)=N(0,I^{-1})\ast M$ with $M=\mathcal{L}(W)$. The covariance is $I^{-1}+\mathrm{Cov}(M)⪰I^{-1}$, equality iff $M={\delta }_0$. $\square$

Connections Master

The Fisher information and Cramér-Rao bound of 45.01.05 are recovered and reinterpreted here: DQM defines the information as the covariance of the $L^2$ score without assuming pointwise density differentiation, and the same $I({\theta }_0)$ becomes the covariance of the central sequence and the curvature of the limit experiment, so the finite-sample lower bound of that unit reappears as the variance $I({\theta }_0{)}^{-1}$ of the Gaussian shift, now provably attainable in the convolution and minimax senses.

The asymptotic normality and efficiency of the MLE 45.04.03 is the estimator-level shadow of LAN: the score expansion that proves $\sqrt{n}({\hat{\theta }}_n-{\theta }_0)\stackrel{d}{\to }N(0,I^{-1})$ there is the same linearisation of the log-likelihood that proves the LAN expansion here, and the efficiency claim there — that $I^{-1}$ is the optimal regular asymptotic variance — is exactly Hájek's convolution theorem read off the Gaussian shift; the superefficiency and Hodges example of that unit are dissolved by the local-asymptotic-minimax bound.

The consistency and stochastic-convergence machinery of 45.04.01 supplies the $o_{\mathbb{P}}$ calculus that controls the root-density remainder, the triangular-array weak law that pins the quadratic term to the constant curvature, and the Slutsky/continuous-mapping arguments that assemble the central sequence; contiguity, the load-bearing change-of-measure tool of the limit-experiment theory, is the asymptotic refinement of those convergence modes.

Contiguity and Le Cam's lemmas 45.04.07 are the direct sequel: the LAN expansion of this unit is precisely the hypothesis under which the first lemma gives mutual contiguity of the local alternatives, the second computes the limit law of the log-ratio under the alternative, and the third shifts an arbitrary limit law — the toolkit that converts the null-distribution LAN expansion into alternative-distribution statements and powers the power calculations downstream.

The convolution and local-asymptotic-minimax bounds 45.04.08 are stated and proved within the Gaussian shift experiment that LAN produces: every regular estimator's limit law factors as $N(0,I^{-1})$ convolved with independent noise, and every estimator's local-minimax risk is bounded below by the shift's, so the efficiency theory of the whole chapter is a corollary of the single convergence-of-experiments statement established here.

The chi-squared limits of the Wald, score, and likelihood-ratio tests 45.04.09 are the testing face of LAN: under contiguous local alternatives the three statistics have non-central chi-squared limits with non-centrality $h^{\mathsf{T}}I({\theta }_0)h$, computed by transporting the null LAN expansion to the alternative via the third lemma, and the asymptotic power of any test is bounded by the most powerful test in the Gaussian shift, the Neyman-Pearson test for a normal mean.

Historical & philosophical context Master

The local-experiment programme is due to Lucien Le Cam, who introduced locally asymptotically normal families in his 1960 monograph Locally asymptotically normal families of distributions ^{[Le Cam 1960]}, defining the LAN condition through the quadratic expansion of the log-likelihood ratio and proving the asymptotic sufficiency of the central sequence. Le Cam's deeper move was to make the experiment — the entire family of measures — the object of study and to introduce a distance (the deficiency) in which sequences of experiments converge; the Gaussian shift is then literally the limit of a smooth model under local rescaling, and asymptotic optimality becomes a property of that limit, transported back by contiguity arguments he systematised as the three lemmas ^{[Le Cam 1986]}.

The efficiency theory in the LAN framework was completed by Jaroslav Hájek, whose 1970 convolution theorem ^{[Hájek 1970]} showed that any regular estimator's limit law factors as $N(0,I^{-1})$ convolved with independent noise, and whose 1972 local asymptotic minimax theorem ^{[Hájek 1972]} removed the regularity hypothesis by a worst-case-over-neighbourhoods criterion, resolving the superefficiency paradox that Le Cam had exhibited in 1953. The clean i.i.d. theory, built on differentiability in quadratic mean as the right regularity condition — derivative-free, accommodating the Laplace family and other non-smooth densities that pointwise differentiation excludes — is the modern textbook synthesis of Le Cam and Hájek, with the quadratic-mean differentiability concept tracing to Le Cam's insistence that the natural map to differentiate is the $L^2$ root-density rather than the density or its logarithm.

Bibliography Master

@book{vanderVaart1998,
  author    = {van der Vaart, Aad W.},
  title     = {Asymptotic Statistics},
  publisher = {Cambridge University Press},
  series    = {Cambridge Series in Statistical and Probabilistic Mathematics},
  year      = {1998}
}

@book{LeCamYang2000,
  author    = {Le Cam, Lucien and Yang, Grace Lo},
  title     = {Asymptotics in Statistics: Some Basic Concepts},
  edition   = {2},
  series    = {Springer Series in Statistics},
  publisher = {Springer},
  year      = {2000}
}

@book{LeCam1986,
  author    = {Le Cam, Lucien},
  title     = {Asymptotic Methods in Statistical Decision Theory},
  series    = {Springer Series in Statistics},
  publisher = {Springer},
  year      = {1986}
}

@article{LeCam1960,
  author  = {Le Cam, Lucien},
  title   = {Locally asymptotically normal families of distributions},
  journal = {University of California Publications in Statistics},
  volume  = {3},
  year    = {1960},
  pages   = {37--98}
}

@article{Hajek1970,
  author  = {H\'ajek, Jaroslav},
  title   = {A characterization of limiting distributions of regular estimates},
  journal = {Zeitschrift f\"ur Wahrscheinlichkeitstheorie und Verwandte Gebiete},
  volume  = {14},
  year    = {1970},
  pages   = {323--330}
}

@inproceedings{Hajek1972,
  author    = {H\'ajek, Jaroslav},
  title     = {Local asymptotic minimax and admissibility in estimation},
  booktitle = {Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability},
  volume    = {1},
  publisher = {University of California Press},
  year      = {1972},
  pages     = {175--194}
}

@article{LeCam1953,
  author  = {Le Cam, Lucien},
  title   = {On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates},
  journal = {University of California Publications in Statistics},
  volume  = {1},
  year    = {1953},
  pages   = {277--330}
}