45.01.03 · mathematical-statistics / 01-decision-theory-estimation

Ancillarity, Completeness, and Basu's Theorem

shipped3 tiersLean: none

Anchor (Master): Lehmann & Casella 1998 Theory of Point Estimation 2nd ed. (Springer) Ch. 1 §6-§7, Ch. 2 §1; Basu 1955 On statistics independent of a complete sufficient statistic (Sankhyā 15); Lehmann & Romano 2005 Testing Statistical Hypotheses 3rd ed. (Springer) §4.3

Intuition Beginner

You measure the heights of a small group of people to learn the average height in a population. Two numbers fall out of the data: the sample average, which tracks the unknown population average, and the spread of the heights around their own average. The spread is interesting. If you slid every height up by ten centimetres — pretending the whole population were taller — the average would move, but the spread would stay exactly the same. The spread does not know where the population average sits.

A summary whose distribution does not move when the unknown quantity moves is called an ancillary statistic. It is a quantity the experiment produces that carries no direct information about the thing you are chasing. The spread of heights around their own average is ancillary to the population average: it is pure scatter, blind to the level.

This sounds useless, but it is the opposite. Once you have squeezed all the information about the unknown into one faithful summary — a sufficient statistic, the idea from the previous unit — the leftover ancillary scatter ought to be unrelated to that summary. If the summary already holds every drop of information, the ancillary noise can have nothing to say about it.

Basu's theorem makes that hunch a fact, under one extra condition called completeness. Completeness says the faithful summary is not carrying any hidden quantity that secretly averages to zero — there is no redundant direction in it. When a summary is both faithful (sufficient) and non-redundant (complete), Basu's theorem guarantees it is statistically independent of every ancillary quantity. The payoff is concrete: you can prove that the sample average and the sample spread are independent for normal data without computing a single integral, just by naming one as complete-sufficient and the other as ancillary.

Visual Beginner

Picture the data as producing two dials. The left dial is the informative summary — for height data, the sample average — and its needle swings as the unknown population average changes. The right dial is the ancillary summary — the spread around the sample average — and its needle's behaviour is frozen with respect to that unknown: shifting everyone's height up by the same amount leaves the right dial's whole pattern of wobble unchanged.

Basu's theorem is the claim that, when the left dial is faithful and non-redundant, the two needles wobble independently: knowing the right dial's reading tells you nothing about the left dial's reading, and the other way around.

Summary	Moves with the unknown level?	Role
Sample average	yes — its whole distribution shifts	informative (sufficient)
Spread around the average	no — distribution is frozen in the level	ancillary
The two together	independent (Basu), once the average is complete	the punchline

The picture is the whole lesson: a frozen-against-the-unknown dial (ancillary) and a faithful non-redundant dial (complete sufficient) turn independently of each other.

Worked example Beginner

We check independence on the smallest informative case. Take two independent draws $X_{1}, X_{2}$ from a population centred at an unknown level $θ$ , where each draw equals $θ$ plus a fair coin's worth of noise: the noise is $+ 1$ or $- 1$ , each with chance one half. So each $X_{i}$ is $θ + 1$ or $θ - 1$ .

Form two summaries: the average $M = (X_{1} + X_{2}) /2$ and the gap $D = X_{1} - X_{2}$ .

Step 1. The gap is ancillary. Compute $D = X_{1} - X_{2} = (θ + noise_{1}) - (θ + noise_{2})$ ; the two $θ$ 's cancel, so $D = noise_{1} - noise_{2}$ , which is $- 2$ , $0$ , or $+ 2$ with chances $1/4$ , $1/2$ , $1/4$ . None of these chances mention $θ$ : the gap's distribution is frozen against the level, so $D$ is ancillary.

Step 2. List the four equally likely outcomes. Each noise pair $(\pm 1, \pm 1)$ has chance $1/4$ :

$(noise_{1}, noise_{2})$	$M = θ + (noise_{1} + noise_{2}) /2$	$D$
$(+ 1, + 1)$	$θ + 1$	$0$
$(+ 1, - 1)$	$θ$	$+ 2$
$(- 1, + 1)$	$θ$	$- 2$
$(- 1, - 1)$	$θ - 1$	$0$

Step 3. Check independence directly. The event $D = 0$ has chance $1/2$ (two of four rows). Given $D = 0$ , the average $M$ is $θ + 1$ or $θ - 1$ , each with chance one half. Given $D = + 2$ , the average is exactly $θ$ . So the value of $D$ tells you which values $M$ can take — wait, that looks like dependence. Recompute the joint table honestly: the pair $(M, D)$ takes four distinct values $(θ + 1, 0)$ , $(θ, 2)$ , $(θ, - 2)$ , $(θ - 1, 0)$ , each with chance $1/4$ . Here $M$ and $D$ are in fact dependent, because this discrete model has a complete sufficient statistic only after more structure is added.

What this tells us: ancillarity of $D$ is real, but independence needs the completeness of the informative summary, which this bare two-point model lacks. The clean independence appears for the normal model, worked in full at the Intermediate tier, where the sample mean is complete sufficient and Basu's theorem applies without computation.

Check your understanding Beginner

Formal definition Intermediate+

Fix a measurable space $(X, A)$ and a statistical model $P = {P_{θ} : θ \in Θ}$ of probability measures on it, in the sense of 45.01.02. Probability spaces, laws, and independence are taken from 37.01.01; the expectation $E_{θ} [\cdot]$ is the integral against $P_{θ}$ of 26.03.01. A statistic is a measurable map $T : X \to (T, B)$ , with law $P_{θ}^{T} = T_{*} P_{θ}$ .

Definition (ancillary statistic). A statistic $A : X \to (S, C)$ is ancillary for $P$ if its law $P_{θ}^{A} = A_{*} P_{θ}$ is the same for every $θ \in Θ$ . It is first-order ancillary if only $E_{θ} [A]$ is constant in $θ$ . An ancillary statistic carries no marginal information about $θ$ ; its interest is conditional, through Fisher's conditionality principle: inference about $θ$ should be carried out in the conditional model given the observed value of an ancillary configuration statistic, because the ancillary indexes which subexperiment was effectively run ^{[Fisher 1934]}.

Definition (complete statistic). A statistic $T$ is complete for $P$ if, for every measurable $g : T \to R$ with $E_{θ} [g (T)]$ defined for all $θ$ ,

E_{θ} [g (T)] = 0 for all θ \in Θ ⟹ g (T) = 0 P_{θ} -a.s. for all θ .

Equivalently, the family of laws ${P_{θ}^{T}}$ admits no non-zero unbiased estimator of zero. The statistic is bounded complete if the implication holds for every bounded measurable $g$ . Completeness is a property of the family ${P_{θ}^{T}}$ , not of a single law; it asserts the family is rich enough that no non-zero function of $T$ can average to zero everywhere — there is no redundant direction in $T$ .

Definition (independence of statistics). Statistics $T$ and $A$ are independent under $P_{θ}$ if $σ (T)$ and $σ (A)$ are independent $σ$ -fields under $P_{θ}$ , i.e. $P_{θ} (T \in B, A \in C) = P_{θ} (T \in B) P_{θ} (A \in C)$ for all $B \in B$ , $C \in C$ , in the sense of 37.01.01. Basu's theorem concerns the case where this holds simultaneously for every $θ$ .

The three notions interlock through minimal sufficiency of 45.01.02. A minimal sufficient statistic is the coarsest faithful summary; an ancillary statistic is, informally, a "complementary" direction carrying no information about $θ$ . A complete sufficient statistic is automatically minimal sufficient (proved below), so completeness is a strengthening of minimality.

Counterexamples to common slips Intermediate+

Sufficient does not imply complete. For an i.i.d. normal sample with known variance, $\overset{ˉ}{X}$ is complete sufficient; but in the location-only uniform family $Unif (θ - 1/2, θ + 1/2)$ the minimal sufficient statistic $(min_{i} X_{i}, max_{i} X_{i})$ is not complete, because the range $max_{i} X_{i} - min_{i} X_{i}$ has $θ$ -free mean and so $g = range - E [range]$ is a non-zero unbiased estimator of zero.
Ancillary is not "uninformative once conditioned". An ancillary statistic has constant marginal law but can be highly informative about the precision of an estimator. In a regression with random design, the design matrix is ancillary for the slope yet determines the conditional variance of the least-squares estimator — the heart of the conditionality principle.
Completeness is needed for Basu, not just sufficiency. Independence of a sufficient statistic from an ancillary can fail when the sufficient statistic is not complete; the discrete two-point model of the Beginner worked example shows the dependence directly. The completeness hypothesis is not decorative.
Bounded completeness is strictly weaker. There are families that are boundedly complete but not complete; bounded completeness already suffices for the similar-tests application (Lehmann-Romano §4.3) and for many uses of Basu's theorem restricted to bounded functions ^{[Lehmann Romano 2005]}.

Key theorem with proof Intermediate+

The organising result is Basu's theorem. We state it for a complete sufficient statistic and prove it from the two defining properties; the proof is the cleanest illustration of why completeness is exactly the missing ingredient.

Theorem (Basu, 1955). Let $P = {P_{θ} : θ \in Θ}$ be a statistical model, let $T$ be a sufficient statistic that is also complete for $P$ , and let $A$ be an ancillary statistic for $P$ . Then $T$ and $A$ are independent under $P_{θ}$ for every $θ \in Θ$ .

Proof. It suffices to show that for every measurable set $C$ in the range of $A$ ,

P_{θ} (A \in C ∣ T) = P_{θ} (A \in C) P_{θ} -a.s., for every θ,

because the right side then does not depend on $T$ , giving independence of $σ (T)$ and $σ (A)$ .

Fix a measurable set $C$ . Define two functions. First, the unconditional probability

p_{C} := P_{θ} (A \in C) .

Because $A$ is ancillary, $p_{C}$ does not depend on $θ$ : it is a fixed number.

Second, the conditional probability given $T$ . Because $T$ is sufficient, the conditional distribution of the data given $T$ does not depend on $θ$ , so a version of

ψ (T) := P_{θ} (A \in C ∣ T) = E_{θ} [1 {A \in C} ∣ T]

may be chosen as a single function of $T$ that is free of $θ$ . (This is the defining property of sufficiency from 45.01.02: the conditional law of the sample, hence of any function $1 {A \in C}$ of it, given $T$ is parameter-free.)

Now consider the difference $g (T) := ψ (T) - p_{C}$ , a fixed measurable function of $T$ . Take its expectation under $P_{θ}$ , using the tower property:

E_{θ} [g (T)] = E_{θ} [ψ (T)] - p_{C} = E_{θ} [E_{θ} [1 {A \in C} ∣ T]] - p_{C} = P_{θ} (A \in C) - p_{C} = p_{C} - p_{C} = 0.

This holds for every $θ \in Θ$ . Since $T$ is complete and $g$ is a function of $T$ with $E_{θ} [g (T)] = 0$ for all $θ$ , completeness forces

g (T) = 0 P_{θ} -a.s., i.e. ψ (T) = p_{C} P_{θ} -a.s.

Thus $P_{θ} (A \in C ∣ T) = p_{C} = P_{θ} (A \in C)$ almost surely, for every $θ$ and every measurable $C$ . Hence $T$ and $A$ are independent under each $P_{θ}$ . $□$

Bridge. Basu's theorem builds toward the distributional calculations that pervade estimation and testing, and appears again in the Lehmann-Scheffé uniqueness of minimum-variance unbiased estimators 45.01.06, where the completeness used to invoke Basu is the very same completeness that makes the conditional estimator unique. The foundational reason the proof works is that sufficiency converts the conditional probability into a $θ$ -free function of $T$ while ancillarity pins the unconditional probability to a $θ$ -free constant; their difference is then an unbiased estimator of zero, and completeness is exactly the hypothesis that kills it. This is exactly the three-property interplay — sufficiency, ancillarity, completeness — read off a single equation $E_{θ} [ψ (T) - p_{C}] = 0$ . The argument generalises verbatim from indicator functions to bounded measurable functions of $A$ , so bounded completeness suffices, and putting these together, the central insight is that completeness is the algebraic non-redundancy condition that turns "no marginal information" (ancillarity) into "no conditional information either" (independence); the bridge is that this same non-redundancy is what makes the natural statistic of a full-rank exponential family — proved complete in the Advanced section — the universal supplier of complete sufficient statistics throughout the theory.

Exercises Intermediate+

Exercise 2 (easy, symbolic).

For $X_{1}, \dots, X_{n}$ i.i.d. $N (θ, 1)$ , prove $\overset{ˉ}{X}$ is complete by reducing completeness to the vanishing of $E_{θ} [g (\overset{ˉ}{X})]$ and citing the normal location family's completeness.

Hint

$\overset{ˉ}{X} \sim N (θ, 1/ n)$ ; if $\int g (t) e^{- n (t - θ)^{2} /2} d t = 0$ for all $θ$ , what does the Laplace transform say?

Answer

$\overset{ˉ}{X} \sim N (θ, 1/ n)$ . Suppose $E_{θ} [g (\overset{ˉ}{X})] = \int g (t) n /2 π e^{- n (t - θ)^{2} /2} d t = 0$ for all $θ$ . Expanding the exponent, $e^{- n (t - θ)^{2} /2} = e^{- n t^{2} /2} e^{n θ t} e^{- n θ^{2} /2}$ , so the condition is $\int [g (t) e^{- n t^{2} /2}] e^{n θ t} d t = 0$ for all $θ \in R$ . The left side is (a constant times) the bilateral Laplace transform of the function $g (t) e^{- n t^{2} /2}$ evaluated at $n θ$ ; vanishing on an open set of arguments forces $g (t) e^{- n t^{2} /2} = 0$ a.e. by the uniqueness theorem for Laplace transforms, hence $g = 0$ a.e. So $\overset{ˉ}{X}$ is complete. Rubric: full credit for the Gaussian factorisation, the Laplace-transform identification, and the uniqueness conclusion.

Exercise 3 (medium, short-answer).

Use Basu's theorem to prove that for $X_{1}, \dots, X_{n}$ i.i.d. $N (θ, 1)$ , the sample mean $\overset{ˉ}{X}$ and the sample variance $S^{2}$ are independent. State which statistic plays which role.

Hint

Exercise 1 made $S^{2}$ ancillary; Exercise 2 made $\overset{ˉ}{X}$ complete sufficient. Apply the theorem.

Answer

With variance known equal to $1$ , $\overset{ˉ}{X}$ is sufficient for $θ$ (factorization: the joint density is $\propto e^{- \frac{n}{2} (\overset{x}{ˉ} - θ)^{2}} \cdot e^{- \frac{1}{2} \sum (x_{i} - \overset{x}{ˉ})^{2}}$ , the first factor a function of $\overset{x}{ˉ}$ and $θ$ ) and complete (Exercise 2). By Exercise 1, $S^{2}$ is ancillary. Basu's theorem applies with $T = \overset{ˉ}{X}$ (complete sufficient) and $A = S^{2}$ (ancillary), so $\overset{ˉ}{X}$ and $S^{2}$ are independent for every $θ$ . Rubric: full credit for assigning $\overset{ˉ}{X}$ = complete sufficient, $S^{2}$ = ancillary, and invoking Basu — no integral needed.

Exercise 4 (medium, symbolic).

Let $X_{1}, \dots, X_{n}$ be i.i.d. $Unif (0, θ)$ , $θ > 0$ . Show $T = max_{i} X_{i}$ is complete and that $R = X_{1} / max_{i} X_{i}$ is ancillary; conclude $max_{i} X_{i}$ and $X_{1} / max_{i} X_{i}$ are independent.

Hint

For completeness, $T / θ$ has a $θ$ -free Beta-type density; for ancillarity, scale all $X_{i}$ by $1/ θ$ .

Answer

The density of $T = max_{i} X_{i}$ is $f_{θ} (t) = n t^{n - 1} / θ^{n}$ on $[0, θ]$ . If $E_{θ} [g (T)] = \int_{0}^{θ} g (t) n t^{n - 1} θ^{- n} d t = 0$ for all $θ$ , then $\int_{0}^{θ} g (t) t^{n - 1} d t = 0$ for all $θ > 0$ ; differentiating in $θ$ gives $g (θ) θ^{n - 1} = 0$ , so $g \equiv 0$ . Thus $T$ is complete (and sufficient, by factorization $\prod_{i} θ^{- 1} 1 {x_{i} \leq θ} = θ^{- n} 1 {max_{i} x_{i} \leq θ}$ ). Writing $X_{i} = θ U_{i}$ with $U_{i} \sim Unif (0, 1)$ , $R = X_{1} / max_{i} X_{i} = U_{1} / max_{i} U_{i}$ , whose law is $θ$ -free, so $R$ is ancillary. Basu gives $max_{i} X_{i} ⊥ X_{1} / max_{i} X_{i}$ . Rubric: full credit for completeness via the integral/derivative argument, the scaling proof of ancillarity, and the Basu conclusion.

Exercise 6 (medium, short-answer).

Explain Fisher's conditionality principle using the mixture-of-experiments example: a coin decides whether you measure $θ$ with a precise instrument (variance $1$ ) or a crude one (variance $100$ ). Identify the ancillary statistic and say why conditioning on it is the right inference.

Hint

Which observed quantity has a distribution free of $θ$ , yet tells you how precise your single measurement was?

Answer

The coin outcome $A$ (which instrument was used) has a fixed $1/2$ - $1/2$ law free of $θ$ , so it is ancillary. Yet $A$ determines the precision of the realised measurement: given $A = precise$ the data are $N (θ, 1)$ , given $A = crude$ they are $N (θ, 100)$ . The conditionality principle says inference should report the precision of the experiment that was actually run, i.e. condition on $A$ , rather than average the two variances into an unconditional report that overstates precision on crude days and understates it on precise days. The ancillary indexes the realised subexperiment; conditioning recovers the relevant information ^{[Fisher 1934]}. Rubric: full credit for naming $A$ ancillary, noting it fixes precision, and stating the conditional inference.

Exercise 7 (hard, symbolic).

Prove that a complete sufficient statistic is minimal sufficient (Bahadur's theorem), assuming a minimal sufficient statistic $U$ exists.

Hint

A minimal sufficient $U$ is a function of every sufficient statistic, so $U = f (T)$ ; show $T$ is a function of $U$ by applying completeness to $g (T) = T - E [T ∣ U]$ (for a bounded coordinate transform).

Answer

Let $T$ be complete sufficient and $U$ minimal sufficient. By minimality $U = f (T)$ for some measurable $f$ (minimal sufficiency is a function of every sufficient statistic). To get the converse, it suffices to show $T$ is $σ (U)$ -measurable. Replace $T$ by a bounded one-to-one transform $\tilde{T} = arctan (T)$ (coordinatewise), still complete and sufficient with $σ (\tilde{T}) = σ (T)$ . Since $U$ is sufficient, $h (U) := E_{θ} [\tilde{T} ∣ U]$ is $θ$ -free; set $g (T) := \tilde{T} - h (f (T))$ , a bounded function of $T$ . By the tower property and $U = f (T)$ , $E_{θ} [g (T)] = E_{θ} [\tilde{T}] - E_{θ} [h (U)] = E_{θ} [\tilde{T}] - E_{θ} [E_{θ} [\tilde{T} ∣ U]] = 0$ for all $θ$ . Completeness of $T$ forces $g (T) = 0$ a.s., i.e. $\tilde{T} = h (U)$ a.s., so $\tilde{T}$ (hence $T$ ) is a function of $U$ . Combined with $U = f (T)$ , $T$ and $U$ generate the same $σ$ -field, so $T$ is minimal sufficient. Rubric: full credit for $U = f (T)$ from minimality, the bounded transform, the completeness application to $\tilde{T} - E [\tilde{T} ∣ U]$ , and the two-way functional dependence.

Exercise 8 (hard, symbolic).

Exhibit a model in which a sufficient statistic is not complete and a sufficient statistic is not independent of an ancillary, demonstrating the necessity of completeness in Basu's theorem. Use $Unif (θ - 1/2, θ + 1/2)$ .

Hint

The minimal sufficient statistic is $(X_{(1)}, X_{(n)}) = (min, max)$ ; the range $X_{(n)} - X_{(1)}$ is ancillary. Show the range and the midrange are dependent.

Answer

For the location-uniform family, the minimal sufficient statistic is $T = (X_{(1)}, X_{(n)})$ , the min and max, sufficient by factorization but not complete: the range $W = X_{(n)} - X_{(1)}$ has $θ$ -free distribution (shift-invariant), so $g (T) := W - E [W]$ satisfies $E_{θ} [g (T)] = 0$ for all $θ$ yet $g (T) \neq \equiv 0$ — a non-zero unbiased estimator of zero, defeating completeness. Now $W$ is ancillary and the midrange $V = (X_{(1)} + X_{(n)}) /2$ is (the natural location estimator), but $V$ and $W$ are dependent: the support constraint $V \pm W /2 \in [θ - 1/2, θ + 1/2]$ ties them, e.g. a large range $W$ near $1$ forces $V$ close to $θ$ , while small $W$ permits $V$ to vary more. Concretely the conditional density of $V$ given $W = w$ is uniform on an interval of length $1 - w$ , which depends on $w$ , so $V \neq ⊥ W$ . Thus the sufficient $T$ (carrying $V$ ) is not independent of the ancillary $W$ — precisely because $T$ is not complete. This is the converse-illustration: completeness is indispensable in Basu's theorem. Rubric: full credit for the incompleteness via the range, the ancillarity of $W$ , and the explicit conditional-length- $(1 - w)$ dependence of $V$ on $W$ .

Advanced results Master

The decisive structural fact is that the natural sufficient statistic of a full-rank exponential family is complete, which makes exponential families the universal source of complete sufficient statistics and so the canonical home of Basu's theorem.

Completeness of the full-rank exponential family. Let ${P_{η} : η \in N}$ be a $k$ -parameter exponential family in canonical form, $p_{η} (x) = h (x) exp (⟨ η, T (x)⟩ - A (η))$ as in 45.01.02, and suppose the natural parameter space $N$ contains a non-empty open set in $R^{k}$ (the full-rank condition). Then $T$ is complete. The proof reduces completeness to the uniqueness theorem for the bilateral Laplace transform: if $g$ is a function of $t$ with $E_{η} [g (T)] = 0$ for all $η$ in the open set, then writing $ν = T_{*} (h d μ)$ for the image of the carrier measure, $\int g (t) e^{⟨ η, t ⟩} ν (d t) = 0$ on an open $η$ -set; splitting $g = g^{+} - g^{-}$ produces two finite measures with equal Laplace transforms on an open set, which by analyticity of the transform must coincide, forcing $g = 0$ $ν$ -a.e. The full-rank (open-set) hypothesis is exactly what licenses the analytic-continuation step; a curved exponential family, whose natural parameters lie on a lower-dimensional submanifold of $N$ , generally fails completeness because the $η$ -set has empty interior and the Laplace transform is pinned only along a curve.

Bahadur's theorem and the completeness hierarchy. A complete sufficient statistic is minimal sufficient (Exercise 7), so completeness sits strictly above minimality: complete sufficient $\Rightarrow$ bounded complete sufficient $\Rightarrow$ minimal sufficient, with neither converse holding. The uniform location family $Unif (θ - 1/2, θ + 1/2)$ is minimal sufficient but not bounded complete; there exist boundedly complete families that are not complete, so all three layers are distinct. Completeness is the property that secures uniqueness in the Lehmann-Scheffé theorem: if $T$ is complete sufficient and $φ (T)$ is unbiased for $ψ (θ)$ , then $φ (T)$ is the unique unbiased function of $T$ estimating $ψ$ , hence — by Rao-Blackwell — the unique minimum-variance unbiased estimator. Basu's theorem and Lehmann-Scheffé thus draw on the same completeness, applied to opposite ends: Basu uses it to kill the difference $ψ (T) - p_{C}$ , Lehmann-Scheffé to kill the difference of two unbiased estimators.

Basu's theorem as a computational engine. Once independence of a complete sufficient $T$ from an ancillary $A$ is in hand, moments factor: $E_{θ} [f (T) m (A)] = E_{θ} [f (T)] E [m (A)]$ with the second factor $θ$ -free. This converts hard joint integrals into products. The independence of $\overset{ˉ}{X}$ and $S^{2}$ for the normal model, the derivation of the distribution of Studentised statistics $n \overset{ˉ}{X} / S$ (whose numerator depends on $T = \overset{ˉ}{X}$ and denominator on the ancillary $S$ , giving the $t$ -distribution by a conditioning argument), and the computation of expectations of ratios all run on Basu. In the theory of similar tests, bounded completeness of the sufficient statistic under the null boundary is precisely the condition under which every similar test has Neyman structure — the test statistic conditioned on the sufficient statistic has $θ$ -free null rejection probability — so Basu-type reasoning underlies the construction of uniformly most powerful unbiased tests ^{[Lehmann Romano 2005]}.

The role of the conditionality principle. Ancillarity also carries an inferential, not merely computational, charge. Fisher's recovery-of-information argument holds that when an ancillary $A$ indexes which subexperiment was effectively performed, the relevant Fisher information is the conditional information given $A$ , and unconditional reports can be misleading ^{[Fisher 1934]}. The tension with Basu's theorem is instructive: Basu shows a complete sufficient statistic is independent of any ancillary, so for a complete-sufficient-statistic-based inference the conditioning on an ancillary changes nothing distributionally; the conditionality principle bites precisely in families without a complete sufficient statistic, where the choice of ancillary to condition on is both consequential and, sometimes, non-unique — the famous non-uniqueness of maximal ancillaries.

Synthesis. The foundational reason ancillarity, completeness, and Basu's theorem cohere is that completeness is the algebraic non-redundancy of a sufficient statistic, and it is exactly the hypothesis that upgrades "an ancillary carries no marginal information" into "an ancillary carries no information at all about a complete sufficient statistic." This is exactly the content of the one-line computation $E_{θ} [ψ (T) - p_{C}] = 0 \Rightarrow ψ (T) = p_{C}$ , in which sufficiency supplies the $θ$ -free $ψ (T)$ , ancillarity supplies the $θ$ -free $p_{C}$ , and completeness supplies the kill. Completeness generalises minimal sufficiency — every complete sufficient statistic is minimal, by the same unbiased-estimator-of-zero argument run on $T - E [T ∣ U]$ — and it is dual to the uniqueness it secures in Lehmann-Scheffé, where the same property annihilates the difference of two unbiased estimators rather than the difference $ψ (T) - p_{C}$ . Putting these together, the central insight is that the full-rank exponential family is the universal supplier: its natural statistic is complete by the Laplace-transform uniqueness theorem, so the convexity-and-analyticity machinery of the log-partition function $A$ from 45.01.02 is what ultimately powers Basu's theorem in every standard model, and the bridge is that the same open-set / analytic-continuation fact that makes $A$ real-analytic on $int N$ is what makes $T$ complete — completeness and the cumulant calculus are two faces of the analyticity of the bilateral transform.

Full proof set Master

Basu's theorem and the normal independence are proved above. The remaining Master claims are recorded here.

Proposition 1 (completeness of the full-rank exponential family). Let ${P_{η} : η \in N}$ have densities $p_{η} = h exp (⟨ η, T ⟩ - A (η))$ with respect to $σ$ -finite $μ$ , and suppose $N$ has non-empty interior. Then $T$ is a complete statistic.

Proof. Let $g$ be measurable with $E_{η} [g (T)] = 0$ for all $η \in int N$ . Let $ν = T_{*} (h d μ)$ be the pushforward of the carrier measure $h d μ$ under $T$ , a $σ$ -finite measure on $R^{k}$ . For $η \in int N$ ,

0 = E_{η} [g (T)] = e^{- A (η)} \int_{R^{k}} g (t) e^{⟨ η, t ⟩} ν (d t), so \int g (t) e^{⟨ η, t ⟩} ν (d t) = 0.

Write $g = g^{+} - g^{-}$ for the positive and negative parts and define finite measures $ν^{\pm} (d t) = g^{\pm} (t) ν (d t)$ (finite because $\int g^{\pm} e^{⟨ η_{0}, t ⟩} d ν < \infty$ at any interior $η_{0}$ , the two parts being separately integrable since $\int ∣ g ∣ p_{η_{0}} d μ = E_{η_{0}} ∣ g (T) ∣ < \infty$ ). The hypothesis reads

\int e^{⟨ η, t ⟩} ν^{+} (d t) = \int e^{⟨ η, t ⟩} ν^{-} (d t) for all η \in int N .

Both sides, as functions of complex $η$ with $Re η \in int N$ , are the bilateral Laplace transforms of the finite measures $ν^{\pm}$ ; each is holomorphic on the tube ${Re η \in int N}$ (differentiation under the integral, as for $A$ in 45.01.02). They agree on the real open set $int N$ , hence on the whole tube by analytic continuation, hence the inverse-transform / uniqueness theorem for Laplace transforms of finite measures gives $ν^{+} = ν^{-}$ . Therefore $g^{+} = g^{-}$ $ν$ -a.e., i.e. $g = 0$ $ν$ -a.e., which is $g (T) = 0$ $P_{η}$ -a.s. for every $η$ (the $P_{η}$ -null sets coincide with the $ν$ -null sets on the support). So $T$ is complete. $□$

Proposition 2 (a complete sufficient statistic is minimal sufficient — Bahadur). If $T$ is complete and sufficient and a minimal sufficient statistic exists, then $T$ is minimal sufficient.

Proof. This is Exercise 7; reproduced. Let $U$ be minimal sufficient. By minimality $U$ is a function of the sufficient $T$ : $U = f (T)$ . Replace $T$ by the bounded injective transform $\tilde{T} = arctan (T)$ componentwise, with $σ (\tilde{T}) = σ (T)$ , still complete and sufficient. Since $U$ is sufficient, $h (U) := E_{θ} [\tilde{T} ∣ U]$ has a $θ$ -free version. Set $g (T) = \tilde{T} - h (f (T))$ , bounded and a function of $T$ . Then, using $U = f (T)$ and the tower property, $E_{θ} [g (T)] = E_{θ} [\tilde{T}] - E_{θ} [h (U)] = E_{θ} [\tilde{T}] - E_{θ} [E_{θ} [\tilde{T} ∣ U]] = 0$ for all $θ$ . Completeness forces $g (T) = 0$ a.s., so $\tilde{T} = h (U)$ a.s.: $\tilde{T}$ , and hence $T$ , is a function of $U$ . With $U = f (T)$ this gives $σ (T) = σ (U)$ a.s., so $T$ is minimal sufficient. $□$

Proposition 3 (Basu, the bounded-complete and conditional forms). Let $T$ be sufficient and bounded complete, and let $A$ be ancillary. Then $T$ and $A$ are independent under every $P_{θ}$ . More generally, if $P$ is a sub-family of a larger model and $A$ is ancillary on $P$ while $T$ is complete sufficient on $P$ , the independence holds on $P$ .

Proof. Repeat the Key-theorem argument with the bounded function $m (A) := 1 {A \in C}$ , which is bounded by $1$ . Sufficiency makes $ψ (T) = E_{θ} [m (A) ∣ T]$ a $θ$ -free function of $T$ , bounded by $1$ ; ancillarity makes $p_{C} = E_{θ} [m (A)] = P_{θ} (A \in C)$ a $θ$ -free constant. Then $g (T) := ψ (T) - p_{C}$ is bounded and satisfies $E_{θ} [g (T)] = 0$ for all $θ$ ; bounded completeness gives $g (T) = 0$ a.s. Hence $P_{θ} (A \in C ∣ T) = P_{θ} (A \in C)$ a.s. for every measurable $C$ , which is independence. The relativised statement is the same proof carried out with all expectations and the completeness restricted to the sub-family $P$ . $□$

Proposition 4 (necessity of completeness). There is a model with a sufficient statistic $T$ and an ancillary $A$ that are dependent. Hence the completeness hypothesis in Basu's theorem cannot be dropped.

Proof. Take $X_{1}, \dots, X_{n}$ i.i.d. $Unif (θ - 1/2, θ + 1/2)$ , $n \geq 2$ . The order-statistic pair $T = (X_{(1)}, X_{(n)})$ is minimal sufficient. The range $W = X_{(n)} - X_{(1)}$ is location-invariant, so its law is $θ$ -free: $W$ is ancillary, and $W$ is a function of $T$ . A statistic cannot be independent of a non-degenerate function of itself: $Cov_{θ} (W, W) = Var_{θ} (W) > 0$ , while independence of $T$ and $W$ would force $Cov (1 {W \leq a}, 1 {W \leq a}) = 0$ . Hence $T$ and the ancillary $W$ are dependent. The obstruction is exactly that $T$ is not complete: $g (T) = W - E [W]$ is a non-zero unbiased estimator of zero. $□$

Connections Master

The probability-space and independence foundations of 37.01.01 are what give Basu's theorem its meaning: ancillarity is the constancy across $θ$ of a pushforward law $A_\astP_\theta$ , completeness is a statement about the family of pushforward laws ${P_{θ}^{T}}$ , and the conclusion is independence of the $σ$ -fields $σ (T)$ and $σ (A)$ in the exact measure-theoretic sense defined there — the theorem is a relation among laws and $σ$ -fields, not among numbers.

The random-variable and expectation theory of 26.03.01 supplies the single equation the proof turns on, $E_{θ} [ψ (T) - p_{C}] = 0$ : the tower property of conditional expectation, the linearity of the integral, and the definition of $E_{θ} [g (T)]$ as an integral against $P_{θ}$ are all imported from that unit, and completeness is phrased entirely as a condition on such expectations.

Sufficiency and exponential families 45.01.02 are the direct upstream: Basu's theorem takes the sufficiency notion and the factorization from there, and the completeness of the natural statistic of a full-rank exponential family — proved here via the Laplace-transform uniqueness theorem — is what makes that unit's exponential families the universal supplier of complete sufficient statistics, so the convexity and analyticity of the log-partition function $A$ are the same analytic facts that power completeness.

The Lehmann-Scheffé theorem and uniformly minimum-variance unbiased estimation 45.01.06 consume completeness from the opposite side: where Basu uses completeness to annihilate $ψ (T) - p_{C}$ and obtain independence, Lehmann-Scheffé uses the identical completeness to annihilate the difference of two unbiased functions of $T$ and obtain uniqueness of the minimum-variance unbiased estimator; the two theorems are the two canonical applications of one property.

The decision-theoretic risk framework 45.01.01 is where the payoff lands: Basu-driven independence of $\overset{ˉ}{X}$ and $S^{2}$ is what makes the Studentised statistic's distribution computable and the $t$ -test's risk and similar-test structure analysable, and bounded completeness of the sufficient statistic under a null boundary is the condition for the Neyman-structure characterisation of similar and uniformly most powerful unbiased tests in the testing chapter 45.02.01.

Historical & philosophical context Master

Ancillarity and the recovery of information by conditioning are due to Ronald A. Fisher, who introduced the configuration statistic and the conditional-inference argument in Two new properties of mathematical likelihood (Proc. R. Soc. A 144, 1934, 285) ^{[Fisher 1934]}, arguing that the precision of an estimate should be measured in the conditional model given an ancillary that indexes the realised experiment. Completeness was introduced by Erich Lehmann and Henry Scheffé in their 1950 Sankhyā paper on completeness, similar regions, and unbiased estimation, where it was the property securing uniqueness of unbiased estimators and the construction of similar regions.

Basu's theorem is due to Debabrata Basu in On statistics independent of a complete sufficient statistic (Sankhyā 15, 1955, 377) ^{[Basu 1955]}, a three-page note whose proof is the one given above; Basu later explored the converse direction and the limits of the result, including examples where independence of a sufficient statistic and an ancillary holds without completeness, showing the theorem's hypotheses are sufficient but not necessary. The completeness of exponential families via the Laplace-transform argument and the placement of ancillarity, completeness, and minimal sufficiency into a single hierarchy were systematised by Lehmann and Casella in Theory of Point Estimation ^{[Lehmann Casella 1998]} and, for the testing applications, in Lehmann and Romano's Testing Statistical Hypotheses ^{[Lehmann Romano 2005]}. Raghu Raj Bahadur proved in 1957 that a complete sufficient statistic is minimal sufficient, fixing completeness as the top of the reduction hierarchy.

Bibliography Master

@article{basu1955statistics,
  author  = {Basu, Debabrata},
  title   = {On statistics independent of a complete sufficient statistic},
  journal = {Sankhy\={a}},
  volume  = {15},
  number  = {4},
  pages   = {377--380},
  year    = {1955}
}

@article{fisher1934two,
  author  = {Fisher, Ronald A.},
  title   = {Two new properties of mathematical likelihood},
  journal = {Proceedings of the Royal Society of London A},
  volume  = {144},
  number  = {852},
  pages   = {285--307},
  year    = {1934}
}

@article{lehmannscheffe1950,
  author  = {Lehmann, Erich L. and Scheff\'e, Henry},
  title   = {Completeness, similar regions, and unbiased estimation. Part I},
  journal = {Sankhy\={a}},
  volume  = {10},
  number  = {4},
  pages   = {305--340},
  year    = {1950}
}

@book{lehmanncasella1998theory,
  author    = {Lehmann, Erich L. and Casella, George},
  title     = {Theory of Point Estimation},
  edition   = {2nd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {1998}
}

@book{lehmannromano2005testing,
  author    = {Lehmann, Erich L. and Romano, Joseph P.},
  title     = {Testing Statistical Hypotheses},
  edition   = {3rd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {2005}
}

@book{casellaberger2002statistical,
  author    = {Casella, George and Berger, Roger L.},
  title     = {Statistical Inference},
  edition   = {2nd},
  publisher = {Duxbury},
  year      = {2002}
}

@article{bahadur1957characterization,
  author  = {Bahadur, Raghu Raj},
  title   = {On unbiased estimates of uniformly minimum variance},
  journal = {Sankhy\={a}},
  volume  = {18},
  pages   = {211--224},
  year    = {1957}
}

Prerequisites

37.01.01
26.03.01
45.01.02

Tier anchors

beginner: Casella & Berger 2002 Statistical Inference 2nd ed. (Duxbury) §6.2.3 (ancillary statistics, the intuition) and §6.2.4 (Basu's theorem, worked uses)
intermediate: Casella & Berger 2002 Statistical Inference 2nd ed. (Duxbury) §6.2.2-§6.2.4 (sufficiency, completeness, ancillarity, Basu); Lehmann & Casella 1998 Theory of Point Estimation 2nd ed. (Springer) Ch. 1 §6, Ch. 2 §1
master: Lehmann & Casella 1998 Theory of Point Estimation 2nd ed. (Springer) Ch. 1 §6-§7, Ch. 2 §1; Basu 1955 On statistics independent of a complete sufficient statistic (Sankhyā 15); Lehmann & Romano 2005 Testing Statistical Hypotheses 3rd ed. (Springer) §4.3

References

Casella, G. & Berger, R. L. — Statistical Inference, 2nd ed. (Duxbury, 2002) · Ch. 6 §6.2.2 (completeness), §6.2.3 (ancillary statistics, the conditionality principle), §6.2.4 (Basu's theorem and worked uses)
Lehmann, E. L. & Casella, G. — Theory of Point Estimation, 2nd ed. (Springer, 1998) · Ch. 1 §6-§7 (completeness, completeness of exponential families); Ch. 2 §1 (ancillarity, Basu's theorem, minimal sufficiency and completeness)
Basu, D. — On statistics independent of a complete sufficient statistic (Sankhyā 15, 1955) · pp. 377-380 (statement and proof of Basu's theorem; the independence of a complete sufficient statistic from any ancillary)
Lehmann, E. L. & Romano, J. P. — Testing Statistical Hypotheses, 3rd ed. (Springer, 2005) · §4.3 (completeness, bounded completeness, and the use of Basu's theorem in similar tests)
Fisher, R. A. — Two new properties of mathematical likelihood (Proc. R. Soc. A 144, 1934) · pp. 285-307 (recovery of information by conditioning on an ancillary configuration statistic)

Estimated time

beginner: 18m
intermediate: 46m
master: 86m