45.01.07 · mathematical-statistics / 01-decision-theory-estimation

Equivariance and the Pitman Best-Equivariant Estimator

shipped3 tiersLean: none

Anchor (Master): Lehmann & Casella 1998 Theory of Point Estimation 2nd ed. (Springer) Ch. 3 §1-§4 (invariant decision problems, transformation groups acting on sample and parameter spaces, the maximal invariant and the reduction to orbits, the minimum-risk equivariant estimator for location and scale, the Pitman estimators, and the Hunt-Stein theorem connecting best-invariant to minimax)

Intuition Beginner

Suppose you are measuring the temperature of a room and your thermometer reads in Celsius. A colleague's thermometer reads the same physical room but is calibrated five degrees high — every reading is shifted up by five. You both want to estimate the true temperature. It would be strange if your two estimates disagreed by anything other than exactly those five degrees. If shifting every data point up by five shifts your estimate up by five too, your method respects the symmetry of the problem. That is the equivariance idea: when the data move in a regular way, the estimate should move along with them.

This matters because many estimation problems have a built-in symmetry. The choice of where to put zero on a scale, or what units to measure in, is arbitrary — the answer should not depend on a bookkeeping convention. An estimator that ignores this and gives wildly different answers under a harmless relabelling is suspect. Demanding that the estimate transform in step with the data is a way of building good sense into the recipe before you even look at the numbers.

The payoff is that this single demand often narrows the field down to almost one rule. Among all the methods that shift with the data, there is usually a single best one — the one whose typical error is smallest. For a shift symmetry (a location parameter) this best rule is the Pitman estimator. It works by looking at the shape of the data — how the points sit relative to one another — and then placing the estimate at the average position consistent with that shape. The symmetry does the heavy lifting; the best estimate falls out almost for free.

Visual Beginner

Picture a row of data points on a number line. Now slide the whole row to the right by some amount. A good estimator slides its answer to the right by exactly the same amount: the gaps between the points never changed, only their common position did, so only the position of the estimate should change.

What you do to the data	What an equivariant estimate does
Add 3 to every observation	The estimate goes up by exactly 3
Subtract 10 from every observation	The estimate goes down by exactly 10
Leave the gaps between points unchanged	The error pattern is unchanged — the method works equally well at every location

The picture is the whole lesson: the gaps between observations tell you about the shape of the population, and the common position tells you about the location. An equivariant rule fixes the shape part and lets only the position part move with the data, so it works just as well no matter where on the line the truth happens to be.

Worked example Beginner

You take three independent measurements of an unknown true length, each scattered around the truth. The readings are $4, 7, 10$ (in centimetres). You want to estimate the true length. Suppose the measurement error is symmetric around the truth — equally likely to overshoot or undershoot. The natural location-equivariant estimate is the sample average.

Step 1. Compute the average. The mean of $4, 7, 10$ is $(4 + 7 + 10) /3 = 21/3 = 7$ . So the estimate is $7$ cm.

Step 2. Check that it shifts with the data. Add $5$ to every reading: the data become $9, 12, 15$ . Their average is $(9 + 12 + 15) /3 = 36/3 = 12$ . The estimate went from $7$ to $12$ , an increase of exactly $5$ . The estimate tracked the shift.

Step 3. Look at the gaps. In the original data the gaps are $7 - 4 = 3$ and $10 - 7 = 3$ . After shifting, the data are $9, 12, 15$ with gaps $12 - 9 = 3$ and $15 - 12 = 3$ . The gaps did not change at all. The shift moved the position but left the shape alone.

Step 4. Read off the equivariant structure. Write each reading as "the average, plus its gap from the average": $4 = 7 - 3$ , $7 = 7 + 0$ , $10 = 7 + 3$ . The gaps from the average, $(- 3, 0, 3)$ , are the same before and after any shift. The average $7$ is the only piece that moves.

What this tells us: an equivariant estimate splits the data into a position part (which moves with any shift) and a shape part (which never moves). For symmetric errors the sample average is the equivariant estimate that places the position exactly at the centre of the data. When the errors are not symmetric, the best equivariant estimate adjusts the centre using the shape — that adjustment is the Pitman estimator.

Check your understanding Beginner

Exercise (easy, multiple choice).

An estimator of a location is location equivariant if:

A. Its value never changes when the data change B. Adding a constant $c$ to every observation increases the estimate by exactly $c$ C. Multiplying the data by $c$ multiplies the estimate by $c$ D. It always equals the first observation

Hint

Recall the thermometer: a five-degree shift in every reading should produce a five-degree shift in the estimate.

Answer

B. Location equivariance means the estimate tracks an additive shift one-for-one. Feedback-correct: correct — adding $c$ to all data adds $c$ to the estimate. Feedback-wrong: A describes an estimator that ignores the data; C is scale equivariance, the multiplicative version; D is a particular rule that happens to be equivariant but is not the definition.

Formal definition Intermediate+

Fix a statistical model $P = {P_{θ} : θ \in Θ}$ on a sample space $(X, A)$ , an action space $A_{act}$ , and a loss $L : Θ \times A_{act} \to [0, \infty)$ as in the decision-theoretic framework of 45.01.01.

Definition (invariant decision problem). Let $G$ be a group of measurable bijections $g : X \to X$ . The model is invariant under $G$ if for each $g \in G$ there is an induced map $\overset{g}{ˉ} : Θ \to Θ$ with

X \sim P_{θ} ⟹ g (X) \sim P_{\overset{g}{ˉ} θ},

equivalently $g_{*} P_{θ} = P_{\overset{g}{ˉ} θ}$ (pushforward). The maps $\overset{g}{ˉ}$ form a group $\overset{ˉ}{G}$ , the induced group on $Θ$ , with $\overline{g_{1} g_{2}} = \overset{g}{ˉ}_{1} \overset{g}{ˉ}_{2}$ . The decision problem is invariant if additionally there is an induced group $\tilde{G}$ acting on $A_{act}$ such that the loss is invariant:

L (\overset{g}{ˉ} θ, \tilde{g} a) = L (θ, a) for all θ, a, g .

Definition (equivariant estimator). An estimator $δ : X \to A_{act}$ is equivariant (under $G$ ) if it commutes with the group actions:

δ (g x) = \tilde{g} δ (x) for all x \in X, g \in G .

The class of equivariant estimators is denoted $D_{G}$ . An estimator $δ^{⋆} \in D_{G}$ is the minimum-risk equivariant (MRE) estimator if $R (θ, δ^{⋆}) \leq R (θ, δ)$ for all $θ$ and all $δ \in D_{G}$ , where $R (θ, δ) = E_{θ} [L (θ, δ (X))]$ is the risk of 45.01.01.

Definition (orbit and maximal invariant). The orbit of $θ$ under $\overset{ˉ}{G}$ is ${\overset{g}{ˉ} θ : g \in G}$ ; orbits partition $Θ$ . A statistic $M : X \to M$ is invariant if $M (g x) = M (x)$ for all $g$ , and is a maximal invariant if it is invariant and constant on $G$ -orbits of $X$ only there: $M (x_{1}) = M (x_{2}) ⟹ x_{2} = g x_{1}$ for some $g$ . A maximal invariant indexes the orbits; every invariant statistic is a function of it.

Definition (location model and location equivariance). Let $X = (X_{1}, \dots, X_{n})$ with density $\prod_{i} f (x_{i} - θ)$ , $θ \in R$ , where $f$ is a fixed known density. The group is $G = {g_{c} : g_{c} (x) = (x_{1} + c, \dots, x_{n} + c), c \in R}$ with induced action $\overset{g}{ˉ}_{c} θ = θ + c$ on $Θ = R$ and $\tilde{g}_{c} a = a + c$ on $A_{act} = R$ . The loss is location invariant if $L (θ, a) = ρ (a - θ)$ for some $ρ \geq 0$ ; squared error $ρ (t) = t^{2}$ qualifies. A location-equivariant estimator satisfies $δ (x + c 1) = δ (x) + c$ for all $c$ .

Definition (scale model and scale equivariance). Let $X$ have density $σ^{- n} \prod_{i} f (x_{i} / σ)$ , $σ > 0$ , with group $G = {g_{b} : g_{b} (x) = (b x_{1}, \dots, b x_{n}), b > 0}$ , induced action $\overset{g}{ˉ}_{b} σ = bσ$ , and scale-invariant loss $L (σ, a) = ρ (a / σ)$ (for example $ρ (t) = (t - 1)^{2}$ or $(lo g t)^{2}$ ). A scale-equivariant estimator satisfies $δ (b x) = b δ (x)$ .

Counterexamples to common slips Intermediate+

Equivariance is not the MLE's functional invariance. The maximum-likelihood estimator satisfies $τ (θ) = τ (θ)$ for a reparametrisation $τ$ — invariance under transformations of the parameter, treated in 45.01.04. Equivariance here is invariance under transformations of the data and parameter jointly through a group action. The two coincide for the location MLE under symmetric $f$ , but they are different principles.
An equivariant estimator need not be unbiased, and the MRE is not generally the UMVUE. The Pitman location estimator is equivariant and minimises MSE within the equivariant class, but for asymmetric $f$ it differs from any UMVU estimator of 45.01.06; equivariance and unbiasedness are distinct restrictions of the estimator class.
Constancy of risk requires loss invariance, not just model invariance. If the loss is not invariant under $\tilde{G}$ , an equivariant estimator's risk need not be constant on orbits, and the reduction to a single number fails. Both ingredients are needed.
The group must act transitively on $Θ$ for the risk to collapse to one number. Location and scale groups act transitively on $R$ and $(0, \infty)$ respectively, so the orbit is the whole parameter space and equivariant risk is a single constant — the feature that makes the MRE estimator well posed.

Key theorem with proof Intermediate+

The two organising facts are structural and constructive. The structural fact is that an equivariant estimator has risk constant on the orbits of $\overset{ˉ}{G}$ , so when $\overset{ˉ}{G}$ acts transitively the risk is a single number and "minimum-risk equivariant" is an honest minimisation. The constructive fact is the explicit form of the best location-equivariant estimator — the Pitman estimator ^{[Casella, G. & Berger, R. L. — Statistical Inference, 2nd ed. (Duxbury, 2002)]}.

Theorem (risk of an equivariant estimator is constant on orbits). Let the decision problem be invariant under $G$ with invariant loss, and let $δ \in D_{G}$ . Then $R (\overset{g}{ˉ} θ, δ) = R (θ, δ)$ for every $g \in G$ and every $θ$ . In particular, if $\overset{ˉ}{G}$ acts transitively on $Θ$ , the risk $R (\cdot, δ)$ is constant on $Θ$ .

Proof. Fix $g \in G$ and $θ$ . Using the change of variables $x \mapsto g x$ , which carries $P_{θ}$ to $P_{\overset{g}{ˉ} θ}$ by model invariance, then equivariance of $δ$ , then loss invariance:

R (\overset{g}{ˉ} θ, δ) = E_{\overset{g}{ˉ} θ} [L (\overset{g}{ˉ} θ, δ (X))] = E_{θ} [L (\overset{g}{ˉ} θ, δ (g X))] = E_{θ} [L (\overset{g}{ˉ} θ, \tilde{g} δ (X))] = E_{θ} [L (θ, δ (X))] = R (θ, δ) .

The second equality is the substitution $X^{'} = g X \sim P_{\overset{g}{ˉ} θ}$ read backwards (a $P_{\overset{g}{ˉ} θ}$ -expectation of a function of $X^{'}$ equals the $P_{θ}$ -expectation of the same function of $g X$ ); the third is $δ (g X) = \tilde{g} δ (X)$ ; the fourth is $L (\overset{g}{ˉ} θ, \tilde{g} a) = L (θ, a)$ . If $\overset{ˉ}{G}$ is transitive, every $θ^{'}$ equals $\overset{g}{ˉ} θ_{0}$ for some fixed $θ_{0}$ , so $R (θ^{'}, δ) = R (θ_{0}, δ)$ is one constant. $□$

Theorem (form and optimality of the location-equivariant estimator; the Pitman estimator). In the location model $X$ with density $\prod_{i} f (x_{i} - θ)$ and squared-error loss, an estimator $δ$ is location equivariant if and only if

δ (x) = δ_{0} (x) - w (y), y = (x_{1} - x_{n}, \dots, x_{n - 1} - x_{n}),

for some fixed equivariant $δ_{0}$ (e.g. $δ_{0} = \overset{ˉ}{X}$ ) and some function $w$ of the differences $y$ . Among all such estimators the one minimising the (constant) risk is the Pitman estimator

δ^{⋆} (x) = \frac{\int _{- \infty}^{\infty} u i = 1 \prod n f ( x _{i} - u ) d u}{\int _{- \infty}^{\infty} i = 1 \prod n f ( x _{i} - u ) d u},

the conditional mean of $θ$ under a flat weighting, equivalently $δ^{⋆} (x) = X_{n} - E_{0} [\overline{Z} ∣ Y = y]$ where $Z_{i} = X_{i} - θ$ and the expectation is at $θ = 0$ .

Proof. Characterisation. If $δ (x + c 1) = δ (x) + c$ , set $w (y) := δ_{0} (x) - δ (x)$ ; then $w (x + c 1) = δ_{0} + c - δ - c = w$ , so $w$ depends on $x$ only through shift-invariant functions, i.e. through the differences $y$ , which are a maximal invariant of $G$ . Conversely any $δ_{0} - w (y)$ with $δ_{0}$ equivariant and $w$ a function of $y$ is equivariant since $y$ is shift-invariant.

Optimality. By the orbit-constancy theorem the risk of every equivariant $δ$ is the constant $R (0, δ) = E_{0} [(δ (X) - 0)^{2}] = E_{0} [(δ_{0} (X) - w (Y))^{2}]$ , evaluated at $θ = 0$ . Condition on the maximal invariant $Y$ :

E_{0} [(δ_{0} - w (Y))^{2}] = E_{0} [E_{0} [(δ_{0} - w (Y))^{2} ∣ Y]] .

For each value $y$ the inner expectation is minimised in $w (y)$ by the conditional mean $w^{⋆} (y) = E_{0} [δ_{0} (X) ∣ Y = y]$ (the $L^{2}$ -projection). Hence the best equivariant estimator is $δ^{⋆} = δ_{0} - E_{0} [δ_{0} ∣ Y]$ , the residual of $δ_{0}$ after removing its conditional mean given the differences. Writing the conditional density of $X$ at $θ = 0$ and integrating out the location yields the displayed integral ratio: with the flat (right-Haar) weighting $d u$ on the location group, the posterior of $θ$ given $x$ is proportional to $\prod_{i} f (x_{i} - u)$ , and its mean is $\int u \prod_{i} f (x_{i} - u) d u / \int \prod_{i} f (x_{i} - u) d u$ , the Pitman estimator. $□$

Bridge. This pair builds toward every invariance-based construction in estimation and testing, and appears again in the scale and location-scale problems and in the Hunt-Stein passage to minimaxity in the Advanced section. The foundational reason the programme works is that an invariant problem has a maximal invariant that strips the data down to its orbit-shape, so the only freedom an equivariant rule has is a function of that shape — and the risk being constant on orbits turns optimisation over a function space into a single conditional minimisation. This is exactly the Rao-Blackwell move of 45.01.06 in a new guise: there one conditions on a sufficient statistic to remove variance; here one conditions on the maximal invariant to remove the orbit-position degree of freedom, and the best choice is again a conditional mean. The construction generalises from squared error to any convex invariant loss, where the conditional mean is replaced by the conditional risk-minimiser, and it is dual to the Bayes construction of 45.01.01: the Pitman estimator is the formal Bayes estimator under the improper right-Haar prior on the group. Putting these together, the central insight is that symmetry plus an invariant loss reduces optimal estimation to a conditional expectation given the maximal invariant, with the bridge being that this conditional expectation is computed by integrating the likelihood against the invariant measure of the group.

Exercises Intermediate+

Exercise 3 (medium, symbolic).

Show that any location-equivariant estimator has the form $δ (x) = δ_{0} (x) - w (y)$ where $y = (x_{1} - x_{n}, \dots, x_{n - 1} - x_{n})$ are the differences and $δ_{0}$ is a fixed equivariant estimator.

Hint

Set $w := δ_{0} - δ$ and check $w (x + c 1) = w (x)$ , so $w$ depends only on shift-invariant functions of $x$ .

Answer

Let $δ_{0}$ be any fixed equivariant estimator (e.g. $δ_{0} = \overset{ˉ}{X}$ or $δ_{0} = X_{n}$ ) and put $w (x) := δ_{0} (x) - δ (x)$ . For a shift, $w (x + c 1) = δ_{0} (x + c 1) - δ (x + c 1) = (δ_{0} (x) + c) - (δ (x) + c) = w (x)$ , so $w$ is shift invariant. The differences $y$ are a maximal invariant for the location group, so any shift-invariant function is a function of $y$ , giving $w = w (y)$ and $δ = δ_{0} - w (y)$ . Conversely $δ_{0} - w (y)$ is equivariant since $y$ is unchanged by shifts. Rubric: full credit for the shift-invariance computation, identifying $y$ as a maximal invariant, and the converse.

Exercise 4 (medium, symbolic).

Derive the Pitman location estimator $δ^{⋆} (x) = \frac{\int u \prod _{i} f ( x _{i} - u ) d u}{\int \prod _{i} f ( x _{i} - u ) d u}$ as the minimiser, over functions $w$ of the differences, of the constant risk $E_{0} [(δ_{0} - w (Y))^{2}]$ .

Hint

The risk is constant on orbits, so evaluate at $θ = 0$ and condition on the maximal invariant $Y$ ; the $L^{2}$ -minimiser is the conditional mean.

Answer

By orbit-constancy the equivariant risk equals its value at $θ = 0$ : $R = E_{0} [(δ_{0} (X) - w (Y))^{2}]$ . Tower over $Y$ : $R = E_{0} [E_{0} [(δ_{0} - w (Y))^{2} ∣ Y]]$ . For each fixed $y$ the inner mean-squared term is minimised in $w (y)$ by the conditional mean $w^{⋆} (y) = E_{0} [δ_{0} (X) ∣ Y = y]$ , so $δ^{⋆} = δ_{0} - E_{0} [δ_{0} ∣ Y]$ . Equivalently, $δ^{⋆} (x) = E [θ ∣ x]$ under the flat weighting $d u$ on the location: the joint "density" $\prod_{i} f (x_{i} - u)$ in $u$ gives posterior mean $\int u \prod_{i} f (x_{i} - u) d u / \int \prod_{i} f (x_{i} - u) d u$ . Rubric: full credit for the reduction to $θ = 0$ , the conditional-mean projection, and the integral-ratio identification.

Exercise 6 (medium, symbolic).

State the scale analogue: for the scale family with density $σ^{- n} \prod_{i} f (x_{i} / σ)$ and loss $ρ (a / σ)$ , derive the form of a scale-equivariant estimator and give the best scale-equivariant estimator of $σ$ under the loss $L (σ, a) = (a / σ - 1)^{2}$ .

Hint

A scale-equivariant $δ$ has $δ (b x) = b δ (x)$ , so $δ (x) = δ_{0} (x) v (z)$ with $z$ the scale-invariant ratios $x_{i} / x_{n}$ ; minimise the orbit-constant risk by conditioning on $z$ .

Answer

Equivariance $δ (b x) = b δ (x)$ gives $δ (x) = δ_{0} (x) v (z)$ with $δ_{0}$ a fixed scale-equivariant estimator and $z = (x_{1} / x_{n}, \dots, x_{n - 1} / x_{n})$ the maximal invariant (the ratios). By orbit-constancy evaluate the risk at $σ = 1$ : $R = E_{1} [(δ_{0} v (Z) - 1)^{2}]$ . Conditioning on $Z$ , the inner minimiser of $E_{1} [(δ_{0} v - 1)^{2} ∣ Z]$ over $v (z)$ is $v^{⋆} (z) = E_{1} [δ_{0} ∣ Z] / E_{1} [δ_{0}^{2} ∣ Z]$ , so the best scale-equivariant estimator is $δ^{⋆} = δ_{0} \cdot E_{1} [δ_{0} ∣ Z] / E_{1} [δ_{0}^{2} ∣ Z]$ , equivalently the integral ratio $\int_{0}^{\infty} t^{n} \prod_{i} f (t x_{i}) d t / \int_{0}^{\infty} t^{n + 1} \prod_{i} f (t x_{i}) d t$ after the substitution $t = 1/ u$ — the Pitman scale estimator for this loss. Rubric: full credit for the multiplicative equivariant form, the ratios as maximal invariant, and the conditional-ratio minimiser.

Exercise 7 (hard, symbolic).

For the exponential location family $f (t) = e^{- t} 1 {t \geq 0}$ , so $X_{i} \geq θ$ with density $\prod_{i} e^{- (x_{i} - θ)} 1 {x_{i} \geq θ}$ , find the Pitman estimator of $θ$ under squared error and compare it to the equivariant minimum $X_{(1)}$ .

Hint

$\prod_{i} f (x_{i} - u) = e^{- \sum_{i} (x_{i} - u)}$ for $u \leq x_{(1)}$ and $0$ otherwise; the likelihood in $u$ is $e^{n u}$ truncated at $u \leq x_{(1)}$ .

Answer

For $u \leq x_{(1)}$ , $\prod_{i} f (x_{i} - u) = exp (- \sum_{i} (x_{i} - u)) = e^{n u} e^{- \sum_{i} x_{i}}$ ; for $u > x_{(1)}$ the product is $0$ . The location-likelihood in $u$ is thus proportional to $e^{n u} 1 {u \leq x_{(1)}}$ . The Pitman estimator is the mean of this truncated density:

δ^{⋆} (x) = \frac{\int _{- \infty}^{x_{(1)}} u e ^{n u} d u}{\int _{- \infty}^{x_{(1)}} e ^{n u} d u} = x_{(1)} - \frac{1}{n},

using $\int_{- \infty}^{a} u e^{n u} d u = e^{na} (a / n - 1/ n^{2})$ and $\int_{- \infty}^{a} e^{n u} d u = e^{na} / n$ . So the Pitman estimator shifts the minimum $X_{(1)}$ down by $1/ n$ , correcting the upward bias of $X_{(1)}$ (which has $E_{θ} X_{(1)} = θ + 1/ n$ ). It is equivariant, has the smallest MSE in the equivariant class, and is exactly the bias-corrected minimum. Rubric: full credit for the truncated-exponential likelihood, the two integrals, the result $X_{(1)} - 1/ n$ , and the comparison to $X_{(1)}$ .

Exercise 8 (hard, symbolic).

Show that the Pitman location estimator is the formal Bayes estimator under the improper uniform (right-Haar) prior $d θ$ on the location group, and use this to argue that under regularity it is minimax (the Hunt-Stein conclusion for the location group).

Hint

With prior $d θ$ the posterior of $θ$ given $x$ is proportional to the likelihood $\prod_{i} f (x_{i} - θ)$ ; its mean under squared-error loss is the Bayes estimate. For minimaxity, the location group is abelian hence amenable, so Hunt-Stein applies.

Answer

Under the improper prior $π (d θ) = d θ$ , the joint "density" of $(θ, x)$ is $\prod_{i} f (x_{i} - θ) d θ$ , so the posterior of $θ$ given $x$ is $\propto \prod_{i} f (x_{i} - θ)$ . By the posterior-expected-loss characterisation of 45.01.01, the squared-error Bayes estimate is the posterior mean

δ^{⋆} (x) = \frac{\int θ \prod _{i} f ( x _{i} - θ ) d θ}{\int \prod _{i} f ( x _{i} - θ ) d θ},

which is exactly the Pitman estimator. The improper prior $d θ$ is the right-invariant Haar measure of the (additive, abelian) location group. The Hunt-Stein theorem states that when the group is amenable — abelian groups are amenable — the best invariant estimator is minimax. Hence $δ^{⋆}$ , the best location-equivariant estimator, is minimax for the location problem under the invariant loss. Rubric: full credit for the posterior identification, the right-Haar prior remark, and the amenability-Hunt-Stein minimax argument.

Advanced results Master

The structural theory has three further strata: the formal-Bayes reading of the MRE estimator under the invariant prior, the Hunt-Stein passage from best-invariant to minimax, and the admissibility status of the Pitman estimator, which is where the invariance programme meets the dimension-sensitivity already seen for the UMVU mean in 45.01.06.

The MRE estimator as a generalised Bayes rule under the invariant prior. For a group $G$ acting transitively on $Θ$ with right-invariant Haar measure $ν$ , the minimum-risk equivariant estimator equals the generalised Bayes estimator against the (typically improper) prior $ν$ on $Θ$ , with the conditional minimisation given the maximal invariant being precisely the posterior-expected-loss minimisation of 45.01.01 computed against $ν$ . In the location case $ν = d θ$ gives the Pitman estimator as the posterior mean under the flat prior; in the scale case $ν = d σ / σ$ (the multiplicative Haar measure) gives the Pitman scale estimator. The improper prior is forced by invariance: a proper prior cannot be invariant under a non-compact transitive group, so the invariant analysis is the natural home of these improper-prior Bayes rules ^{[Lehmann, E. L. & Casella, G. — Theory of Point Estimation, 2nd ed. (Springer, 1998)]}.

The Hunt-Stein theorem. Let the decision problem be invariant under a group $G$ that is amenable (a left-invariant mean exists on its bounded functions; abelian and solvable groups, hence the location and scale and location-scale groups, are amenable). Then there is a minimax estimator that is invariant, and consequently the minimum-risk equivariant estimator is minimax. The proof averages an arbitrary estimator over the group using the invariant mean, producing an invariant estimator with no larger maximum risk, then invokes the constancy of equivariant risk on orbits to identify the worst-case risk with the single orbit-value. Amenability is essential and not a technicality: for the full linear group acting on a multivariate normal mean — which is not amenable in the relevant sense — the conclusion fails, and the best invariant estimator (the sample mean) is not minimax-optimal in the strong sense, the analytic shadow of Stein's phenomenon ^{[Lehmann, E. L. & Casella, G. — Theory of Point Estimation, 2nd ed. (Springer, 1998)]}.

Admissibility of the Pitman estimator and the boundary with Stein. In dimension one the Pitman estimator of a location parameter is admissible under squared-error loss for a wide class of $f$ (Stein's 1959 result, refined by Brown's 1966 differential-inequality criterion). The boundary is again dimensional: for a $p$ -dimensional location parameter with $p \geq 3$ the best location-equivariant estimator — which for the Gaussian is the sample mean — is inadmissible by Stein's phenomenon of 45.01.06, dominated by the James-Stein shrinkage estimator, which is not equivariant under the full translation group of $R^{p}$ because shrinkage toward a fixed origin breaks translation symmetry. Equivariance under the location group thus singles out a unique best rule within its class, but that class can itself be uniformly improvable once the dimension is high enough — the same tension between a symmetry-constrained optimum and global admissibility that distinguishes UMVU from admissible estimation ^{[Lehmann, E. L. & Casella, G. — Theory of Point Estimation, 2nd ed. (Springer, 1998)]}.

The location-scale problem and reduction by stages. For the location-scale family $σ^{- n} \prod_{i} f ((x_{i} - μ) / σ)$ the group is the affine group $x \mapsto b x + c 1$ , non-abelian but solvable hence amenable. The maximal invariant is the configuration $z_{i} = (x_{i} - x_{n}) / s$ for a scale statistic $s$ , and the best equivariant estimators of $μ$ and of $σ$ are the affine-group Pitman estimators, computed by integrating the likelihood against the right-Haar measure $d c d σ / σ$ of the affine group. The reduction proceeds in stages — first quotient by translations, then by scalings — and the resulting estimators are minimax by Hunt-Stein since the affine group is amenable ^{[Lehmann, E. L. & Casella, G. — Theory of Point Estimation, 2nd ed. (Springer, 1998)]}.

Synthesis. The foundational reason invariance is a complete estimation principle for transitive groups is that the maximal invariant exhausts the orbit-shape of the data while the group sweeps out the orbit, so an equivariant rule is a function of the maximal invariant carried along the group, and its risk is constant on orbits — the single computation of conditioning on the maximal invariant produces the optimum. This is exactly the Rao-Blackwell projection of 45.01.06 reread with the maximal invariant in place of the sufficient statistic: there sufficiency removes variance by conditioning, here invariance removes the orbit-position freedom by conditioning, and both optima are conditional means. The MRE estimator is dual to the Bayes estimator of 45.01.01: it is the generalised Bayes rule under the right-Haar prior, the unique invariant measure the symmetry permits, so the Pitman estimator is at once frequentist-optimal-within-the-equivariant-class and Bayes-optimal-under-the-invariant-prior. Putting these together with the Hunt-Stein theorem, the central insight is that for an amenable group the symmetry that defines the equivariant class also delivers minimaxity, so best-invariant equals minimax — and the bridge is amenability, whose failure for the high-dimensional translation group is exactly the analytic content of Stein's phenomenon, the boundary where the symmetry-constrained optimum stops being globally admissible.

Full proof set Master

The orbit-constancy of risk and the form-and-optimality of the Pitman location estimator are proved in the Key theorem. The remaining Master claims are recorded here.

Proposition 1 (the differences are a maximal invariant for the location group). For $G = {x \mapsto x + c 1 : c \in R}$ on $R^{n}$ , the map $M (x) = (x_{1} - x_{n}, \dots, x_{n - 1} - x_{n})$ is a maximal invariant.

Proof. Invariance: $M (x + c 1) = ((x_{1} + c) - (x_{n} + c), \dots) = M (x)$ . Maximality: suppose $M (x) = M (x^{'})$ , so $x_{i} - x_{n} = x_{i}^{'} - x_{n}^{'}$ for $i < n$ . Put $c = x_{n}^{'} - x_{n}$ . Then $x_{i}^{'} = x_{i} + (x_{n}^{'} - x_{n}) = x_{i} + c$ for $i < n$ , and $x_{n}^{'} = x_{n} + c$ as well, so $x^{'} = x + c 1 = g_{c} x$ lies in the orbit of $x$ . Hence equal values of $M$ force the same orbit, the maximality condition. $□$

Proposition 2 (the Pitman estimator is the generalised Bayes rule under the flat prior). In the location model under squared-error loss, the Pitman estimator $δ^{⋆} (x) = \int u \prod_{i} f (x_{i} - u) d u / \int \prod_{i} f (x_{i} - u) d u$ is the formal posterior mean of $θ$ under the improper prior $π (d θ) = d θ$ .

Proof. With prior $d θ$ , the joint measure of $(θ, X)$ has "density" $\prod_{i} f (x_{i} - θ)$ with respect to $d θ d x$ . The conditional law of $θ$ given $X = x$ is therefore proportional to $θ \mapsto \prod_{i} f (x_{i} - θ)$ , normalised by $\int \prod_{i} f (x_{i} - u) d u$ (finite under the integrability hypothesis). The squared-error Bayes action is the posterior mean (part (i) of the Key theorem of 45.01.01), namely $\int u \prod_{i} f (x_{i} - u) d u / \int \prod_{i} f (x_{i} - u) d u = δ^{⋆} (x)$ . $□$

Proposition 3 (equivalence of the conditional-mean and integral-ratio forms). With $Z_{i} = X_{i} - θ$ and $Y$ the differences, $δ^{⋆} (x) = X_{n} - E_{0} [\overline{Z} ∣ Y = y]$ equals the integral ratio $\int u \prod_{i} f (x_{i} - u) d u / \int \prod_{i} f (x_{i} - u) d u$ .

Proof. Take $δ_{0} = \overset{ˉ}{X}$ , equivariant since $\overline{x + c 1} = \overset{x}{ˉ} + c$ . By the optimality argument $δ^{⋆} = \overset{ˉ}{X} - E_{0} [\overset{ˉ}{X} ∣ Y]$ . At $θ = 0$ , $X_{i} = Z_{i}$ , and $\overset{ˉ}{X} = \overline{Z}$ , so $δ^{⋆} = X_{n} - E_{0} [\overline{Z} ∣ Y]$ after writing $\overset{ˉ}{X} = X_{n} + (\overline{X} - X_{n})$ and absorbing the shift-invariant part into the conditional mean. To get the integral form, substitute $u = θ$ in the joint density at general $θ$ : the conditional density of $X$ given $Y = y$ at $θ = 0$ is $\prod_{i} f (x_{i}) / \int (\int \prod_{i} f (\cdot))$ , and the conditional mean of the location is computed by integrating $u$ against $\prod_{i} f (x_{i} - u)$ , giving the displayed ratio after normalisation. The two expressions agree because both are the $L^{2}$ -projection of the location onto the $σ$ -field of the maximal invariant. $□$

Proposition 4 (Hunt-Stein for an amenable group, statement and proof sketch with the averaging step explicit). If $G$ is amenable and the problem is invariant with invariant loss, there exists an invariant minimax estimator; hence the MRE estimator is minimax.

Proof. Let $m$ be a left-invariant mean on the bounded functions of $G$ . Given any estimator $δ$ with finite maximum risk, define its group average $δ^{G}$ by averaging the rule's action over $G$ using $m$ (in the location case, $δ^{G} (x) = \int (δ (x - c 1) + c) m (d c)$ in the formal Følner-limit sense). The averaged rule is equivariant: applying $g$ to the data and using left-invariance of $m$ reproduces $\tilde{g} δ^{G} (x)$ . By convexity of the loss and Jensen applied through $m$ , $R (θ, δ^{G}) \leq sup_{g} R (θ, \tilde{g} \cdot δ \circ g) = sup_{g} R (\overset{g}{ˉ}^{- 1} θ, δ)$ , so the maximum risk of $δ^{G}$ does not exceed that of $δ$ . Taking $δ$ to approach the minimax value gives an invariant estimator attaining it. Constancy of equivariant risk on orbits identifies $sup_{θ} R (θ, δ^{G})$ with the single orbit-value, so the MRE estimator — which minimises that value over the equivariant class — is minimax. Amenability is exactly what guarantees the invariant mean $m$ used in the averaging exists; without it the averaging step has no limit. $□$

Proposition 5 (Pitman scale estimator). In the scale model $σ^{- n} \prod_{i} f (x_{i} / σ)$ with loss $L (σ, a) = (a / σ - 1)^{2}$ , the best scale-equivariant estimator is

δ^{⋆} (x) = \frac{\int _{0}^{\infty} u ^{- n - 1} ( \prod _{i} f ( x _{i} / u ) ) u ^{- 1} d u}{\int _{0}^{\infty} u ^{- n - 2} ( \prod _{i} f ( x _{i} / u ) ) u ^{- 1} d u},

the generalised Bayes rule under the scale-Haar prior $d σ / σ$ .

Proof. Scale equivariance gives $δ (x) = δ_{0} (x) v (z)$ with $z$ the ratios; orbit-constancy evaluates the risk at $σ = 1$ , $R = E_{1} [(δ_{0} v (Z) - 1)^{2}]$ . Conditioning on $Z$ and minimising the quadratic in $v (z)$ gives $v^{⋆} (z) = E_{1} [δ_{0} ∣ Z] / E_{1} [δ_{0}^{2} ∣ Z]$ , hence $δ^{⋆} = δ_{0} E_{1} [δ_{0} ∣ Z] / E_{1} [δ_{0}^{2} ∣ Z]$ . Translating into the posterior under the Haar prior $π (d σ) = d σ / σ$ : the joint "density" is $σ^{- n} \prod_{i} f (x_{i} / σ) \cdot σ^{- 1}$ , the posterior of $σ$ given $x$ is $\propto σ^{- n - 1} \prod_{i} f (x_{i} / σ)$ , and the Bayes action under the loss $(a / σ - 1)^{2}$ is $a = E [σ^{- 1} ∣ x] / E [σ^{- 2} ∣ x]$ . Writing $u = σ$ and reading the two posterior moments as the displayed integrals yields the stated ratio. $□$

Connections Master

Decision theory, risk, and admissibility 45.01.01 supplies the entire frame this unit specialises: the loss, risk, and admissibility order are unchanged, invariance is an additional restriction on the estimator class, and the Pitman estimator is literally the generalised Bayes rule under the right-Haar prior, so the posterior-expected-loss characterisation proved there computes the MRE estimator here; the Hunt-Stein theorem then upgrades best-invariance to the minimaxity also defined there.

Rao-Blackwell and Lehmann-Scheffé 45.01.06 are the structural twin of the equivariance reduction: Rao-Blackwell conditions on a sufficient statistic to remove variance, the equivariance argument conditions on the maximal invariant to remove the orbit-position freedom, and both optima are conditional means; moreover the dimensional boundary is shared, since the best location-equivariant estimator of a Gaussian mean is the UMVU sample mean and is inadmissible for dimension three and up by the same Stein phenomenon, the James-Stein dominator being neither unbiased nor translation-equivariant.

Maximum-likelihood estimation and its functional invariance 45.01.04 must be distinguished from the equivariance of this unit: the MLE's invariance is $τ (θ) = τ (θ)$ under reparametrisation of the parameter, a property of the likelihood maximiser, whereas equivariance is commutation with a group acting jointly on data and parameter; for a symmetric location family the location MLE is equivariant and frequently coincides with the Pitman estimator, but the two invariance principles are logically separate.

Sufficiency and exponential families 45.01.02 interact with invariance through the reduction by stages: the maximal invariant of a group is typically computed after reducing to a sufficient statistic, and for location-scale families the configuration statistic (the standardised residuals) is the maximal invariant on which the Pitman estimators are conditional means, so the factorisation theory and the invariance theory compose to give the explicit best-equivariant rules.

Hypothesis testing and most-powerful tests 45.02.01 use the same invariance reduction on the testing side: a uniformly most powerful invariant test is found by reducing to the maximal invariant and applying Neyman-Pearson to its distribution, the testing analogue of the MRE estimator, and the Hunt-Stein theorem there yields minimax tests for amenable groups exactly as it yields minimax estimators here.

Historical & philosophical context Master

The explicit best location-equivariant and scale-equivariant estimators are due to E. J. G. Pitman, in his 1939 Biometrika paper on estimating the location and scale parameters of a continuous population of arbitrary known form ^{[Pitman 1939]}. Pitman derived the estimators as conditional means given the sample configuration and showed they minimise mean squared error among estimators with the appropriate equivariance, predating the decision-theoretic vocabulary that later subsumed them.

The invariance principle was placed inside Abraham Wald's decision-theoretic framework and systematised by Charles Stein, Erich Lehmann, and others through the 1940s and 1950s; the theorem that an invariant problem under an amenable group admits an invariant minimax rule is the Hunt-Stein theorem, from the unpublished work of G. A. Hunt and Charles Stein around 1946 and published accounts by Lehmann and by Kiefer ^{[Lehmann Casella 1998]}. The role of amenability — and the failure of the conclusion for non-amenable groups such as the full linear group on a multivariate normal mean — connects the invariance theory directly to Stein's 1956 inadmissibility result.

The admissibility of the Pitman estimator in one dimension was established by Stein in 1959 and given a sharp differential-inequality criterion by Lawrence Brown in 1966; Brown's analysis tied admissibility of equivariant estimators to the recurrence of an associated diffusion, the same circle of ideas through which the dimension-three threshold of Stein's phenomenon was understood. The standard modern synthesis of invariant estimation, the Pitman estimators, and the Hunt-Stein theorem is Lehmann and Casella's Theory of Point Estimation ^{[Lehmann Casella 1998]} Chapter 3.

Bibliography Master

@article{pitman1939estimation,
  author  = {Pitman, E. J. G.},
  title   = {The estimation of the location and scale parameters of a continuous population of any given form},
  journal = {Biometrika},
  volume  = {30},
  number  = {3/4},
  pages   = {391--421},
  year    = {1939}
}

@book{lehmanncasella1998theory,
  author    = {Lehmann, Erich L. and Casella, George},
  title     = {Theory of Point Estimation},
  edition   = {2nd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {1998}
}

@book{casellaberger2002statistical,
  author    = {Casella, George and Berger, Roger L.},
  title     = {Statistical Inference},
  edition   = {2nd},
  publisher = {Duxbury},
  year      = {2002}
}

@book{ferguson1967mathematical,
  author    = {Ferguson, Thomas S.},
  title     = {Mathematical Statistics: A Decision Theoretic Approach},
  publisher = {Academic Press},
  year      = {1967}
}

@article{brown1966admissibility,
  author  = {Brown, Lawrence D.},
  title   = {On the admissibility of invariant estimators of one or more location parameters},
  journal = {Annals of Mathematical Statistics},
  volume  = {37},
  number  = {5},
  pages   = {1087--1136},
  year    = {1966}
}

@inproceedings{stein1956inadmissibility,
  author    = {Stein, Charles},
  title     = {Inadmissibility of the usual estimator for the mean of a multivariate normal distribution},
  booktitle = {Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability},
  volume    = {1},
  pages     = {197--206},
  publisher = {University of California Press},
  year      = {1956}
}

Prerequisites

45.01.01
45.01.06

Tier anchors

beginner: Casella & Berger 2002 Statistical Inference 2nd ed. (Duxbury) §7.3.4 (the equivariance / invariance principle in point estimation, location equivariance, and the picture of an estimator that moves with the data)
intermediate: Casella & Berger 2002 Statistical Inference 2nd ed. (Duxbury) §7.3.4 (location-equivariant estimators, the form of the location-equivariant estimator as a function of the differences plus an additive constant, and the Pitman estimator minimising mean squared error); Lehmann & Casella 1998 Theory of Point Estimation 2nd ed. (Springer) §3.1-§3.2
master: Lehmann & Casella 1998 Theory of Point Estimation 2nd ed. (Springer) Ch. 3 §1-§4 (invariant decision problems, transformation groups acting on sample and parameter spaces, the maximal invariant and the reduction to orbits, the minimum-risk equivariant estimator for location and scale, the Pitman estimators, and the Hunt-Stein theorem connecting best-invariant to minimax)

References

Casella, G. & Berger, R. L. — Statistical Inference, 2nd ed. (Duxbury, 2002) · §7.3.4 (the equivariance principle, location-equivariant estimators and their form as a function of the differences plus a constant, the Pitman minimum-MSE location estimator, and the contrast with the functional invariance of the MLE)
Lehmann, E. L. & Casella, G. — Theory of Point Estimation, 2nd ed. (Springer, 1998) · Ch. 3 §1-§4 (invariant decision problems and transformation groups, equivariant estimators, the maximal invariant and reduction to orbits, the minimum-risk equivariant location and scale estimators, the Pitman estimators, and the Hunt-Stein theorem relating best invariance to minimaxity)
Pitman, E. J. G. — The estimation of the location and scale parameters of a continuous population of any given form (Biometrika 30, 1939) · pp. 391-421 (the construction of the best location-equivariant and best scale-equivariant estimators as conditional means given the configuration of the sample — the Pitman estimators)
Ferguson, T. S. — Mathematical Statistics: A Decision Theoretic Approach (Academic Press, 1967) · Ch. 4 (invariance in statistical decision problems, induced groups on the parameter and action spaces, invariant loss, and the constancy of the risk of an equivariant rule on orbits)

Estimated time

beginner: 18m
intermediate: 46m
master: 86m