Statistical Decision Theory: Loss, Risk, Admissibility, Minimax, and Bayes Rules

Intuition Beginner

Every time you use data to make a call — estimate a quantity, choose between two explanations, decide whether a part is defective — you are picking a rule that turns observations into an action. Two honest people with the same data can use different rules. Statistical decision theory is the framework for asking which rule is better, and what "better" even means once chance is in the picture.

The first ingredient is a way to score mistakes. A loss says how bad it is to take action $a$ when the truth is $\theta$. Guessing a temperature of 19 when it is really 20 should cost less than guessing 5. A common score is the squared miss, but the framework allows any sensible cost. The loss is the value judgment; everything after it is mathematics.

A rule does not have one loss, it has a whole spread of them, because the data are random. So we average the loss over the randomness in the data, holding the truth $\theta$ fixed. That average is the risk of the rule at $\theta$. A rule with low risk is one whose mistakes are small on average, whatever specific data happen to arrive.

The catch is that risk depends on the unknown truth $\theta$. One rule might be best when $\theta$ is small, another best when $\theta$ is large. There is rarely a single rule that beats all others everywhere. So the theory offers principles for breaking the tie: avoid rules that are beaten outright (keep only the admissible ones), protect against the worst case (minimax), or weigh the possible truths by a prior belief and minimise the weighted average (Bayes). These three principles, and how they secretly fit together, are the whole subject.

Visual Beginner

Figure: a horizontal axis for the unknown truth $\theta$ and a vertical axis for risk. Two risk curves are drawn. Rule A is a flat line at a moderate height — same risk for every $\theta$. Rule B dips very low near the centre but rises steeply at the edges. Where B is below A, B is the better rule; where B is above A, A wins. Neither curve lies entirely below the other, so neither rule dominates: both are admissible. A third dashed curve C sits entirely above A everywhere — C is dominated and therefore inadmissible.

 risk
  |                                          .C (dominated: above A everywhere)
  |   .........................................   <- dashed, always worst
  |
  |----A----------------------------------------  rule A: flat risk (minimax-ish)
  |        \                              /
  |         \         rule B            /
  |          \.____________________.../        B: low in the middle, high at edges
  |
  +------------------------------------------------> theta
       A vs B: neither dominates (curves cross)  -> both admissible
       C vs A: A dominates C (A below C for all theta) -> C inadmissible

Worked example Beginner

You observe one draw $X$ from a population whose unknown mean is $\theta$, with the draw scattered around $\theta$ with variance $4$. You want to estimate $\theta$. Compare two rules: the rule ${\delta }_1(X)=X$ (just report the data), and the shrink-toward-zero rule ${\delta }_2(X)=\frac{1}{2}X$. Score each by squared error, $L(\theta ,a)=(\theta -a{)}^2$.

Step 1. Risk of ${\delta }_1$. The risk is the average squared error. For ${\delta }_1(X)=X$, the error is $X-\theta$, whose average square is the variance, $4$. So the risk is $4$ for every $\theta$ — a flat line.

Step 2. Risk of ${\delta }_2$. Now the error is $\frac{1}{2}X-\theta$. Split it into a typical-offset part and a wobble part. On average $\frac{1}{2}X$ sits at $\frac{1}{2}\theta$, so it is off-target by $\frac{1}{2}\theta$ (this is the bias). The wobble of $\frac{1}{2}X$ has variance $\frac{1}{4}\times 4=1$. The average squared error is bias-squared plus wobble:

$$R(\theta ,{\delta }_2)={\left(\frac{1}{2}\theta \right)}^2+1=\frac{1}{4}{\theta }^2+1.$$

Step 3. Compare. When is ${\delta }_2$ better? Solve $\frac{1}{4}{\theta }^2+1<4$, i.e. ${\theta }^2<12$, i.e. $|\theta |<3.46$. So ${\delta }_2$ wins for truths near zero and loses for truths far out.

What this tells us. Neither rule beats the other everywhere: their risk curves cross. Both are admissible — you cannot discard either on dominance grounds alone. Choosing between them needs an extra principle: if you fear a large $|\theta |$, the flat rule ${\delta }_1$ is safer (minimax); if you believe $\theta$ is probably near zero, the shrinker ${\delta }_2$ is better on average (Bayes).

Check your understanding Beginner

Formal definition Intermediate+

A statistical decision problem is specified by a parameter space $\Theta$, a family of probability measures $\{P_{\theta }:\theta \in \Theta \}$ on a sample space $(\mathcal{X},\mathcal{B})$ (the measure-theoretic substrate of 37.01.01), an action space $\mathcal{A}$, and a loss function $L:\Theta \times \mathcal{A}\to [0,\infty ]$ that is measurable in its arguments. A (non-randomized) decision rule is a measurable map $\delta :\mathcal{X}\to \mathcal{A}$; on observing $X\sim P_{\theta }$ one takes the action $\delta (X)$.

Definition (risk function). The risk of a rule $\delta$ at $\theta$ is the expected loss

$$\boxed{R(\theta ,\delta )={\mathbb{E}}_{\theta }[L(\theta ,\delta (X))]={\int }_{\mathcal{X}}L(\theta ,\delta (x))d{P}_{\theta }(x).}$$

A rule is identified with the function $\theta \mapsto R(\theta ,\delta )$ it induces. For point estimation with $\mathcal{A}=\Theta \subseteq \mathbb{R}$ and squared-error loss $L(\theta ,a)=(\theta -a{)}^2$, the risk is the mean squared error, with the bias-variance decomposition $R(\theta ,\delta )={\mathrm{Var}}_{\theta }(\delta )+({\mathbb{E}}_{\theta }\delta -\theta {)}^2$ ^{[Casella & Berger §7.3]}.

Definition (dominance and admissibility). A rule ${\delta }^{\prime }$ dominates $\delta$ if $R(\theta ,{\delta }^{\prime })\le R(\theta ,\delta )$ for all $\theta \in \Theta$ with strict inequality for at least one $\theta$. A rule $\delta$ is inadmissible if some ${\delta }^{\prime }$ dominates it, and admissible otherwise. Admissibility is a minimal requirement, not a recommendation: it only rules out being beaten outright, and many admissible rules are poor.

Definition (minimax rule). The maximum risk of $\delta$ is ${\sup }_{\theta }R(\theta ,\delta )$. A rule ${\delta }^{\ast }$ is minimax if it minimises the maximum risk,

$$\underset{\theta }{\sup }R(\theta ,{\delta }^{\ast })=\underset{\delta }{\inf }\underset{\theta }{\sup }R(\theta ,\delta )=:\bar{V},$$

the minimax value of the problem. Minimaxity is the conservative, worst-case-protective principle.

Definition (Bayes risk and Bayes rule). Given a prior probability measure $\pi$ on $\Theta$, the Bayes risk of $\delta$ is the prior-averaged risk

$$r(\pi ,\delta )={\int }_{\Theta }R(\theta ,\delta )d\pi (\theta ).$$

A rule ${\delta }_{\pi }$ is a Bayes rule with respect to $\pi$ if it minimises the Bayes risk, $r(\pi ,{\delta }_{\pi })={\inf }_{\delta }r(\pi ,\delta )=:r(\pi )$, the Bayes value. The posterior $\pi (\cdot \mid x)$ obtained by conditioning $\pi$ on $X=x$ (the Bayesian update of 26.07.01) gives the posterior expected loss $\rho (\pi ,a\mid x)={\int }_{\Theta }L(\theta ,a)d\pi (\theta \mid x)$.

Definition (randomized rule and the risk set). A randomized rule $\delta$ chooses an action by drawing from a distribution $\delta (\cdot \mid x)$ on $\mathcal{A}$ after seeing $x$; its risk is $R(\theta ,\delta )={\int }_{\mathcal{X}}{\int }_{\mathcal{A}}L(\theta ,a)\delta (da\mid x)d{P}_{\theta }(x)$. When $\Theta =\{{\theta }_1,\dots ,{\theta }_k\}$ is finite, the risk set is

$$S=\{(R({\theta }_1,\delta ),\dots ,R({\theta }_k,\delta )):\delta \text{ a randomized rule}\}\subseteq {\mathbb{R}}^k,$$

a convex subset of ${\mathbb{R}}^k$ in the sense of 44.01.01: convexity comes from mixing rules, since the $1/2$-$1/2$ random choice between ${\delta }^{\prime }$ and ${\delta }^{\prime \prime }$ has risk vector the midpoint of theirs.

Counterexamples to common slips

• Admissible does not mean good. The constant rule $\delta (x)\equiv c$ (ignore the data, always estimate $c$) is admissible under squared-error loss on $\Theta =\mathbb{R}$: its risk is $0$ at $\theta =c$, so no competitor can be uniformly no-worse without also being $0$ there, which pins it down. A useless rule can be admissible.
• Minimax is not unique and need not be Bayes for an obvious prior. Several rules can share the minimax value; the minimax rule is the Bayes rule for a least favourable prior, which is often supported on the boundary of $\Theta$, not the prior a practitioner would write down.
• A Bayes rule can be randomized only on a null set. When the posterior expected loss has a unique minimiser for almost every $x$, the Bayes rule is essentially non-randomized; randomization buys nothing under strict convexity of the loss. Randomization matters for minimaxity in finite-$\Theta$ problems, not for typical Bayes estimation.

Key theorem with proof Intermediate+

The organising theorem is that the Bayes rule is computed pointwise by minimising posterior expected loss, and that Bayes rules with finite Bayes risk are almost the cheapest possible certificate of admissibility.

Theorem (Bayes rule via posterior expected loss; admissibility of Bayes rules). Fix a prior $\pi$ with finite Bayes value $r(\pi )<\infty$.

(i) (Posterior characterization.) Suppose the joint law of $(\theta ,X)$ admits the disintegration $d\pi (\theta )d{P}_{\theta }(x)=dm(x)d\pi (\theta \mid x)$ with marginal $m$. If for $m$-almost every $x$ the action ${\delta }_{\pi }(x)$ minimises the posterior expected loss $a\mapsto \rho (\pi ,a\mid x)={\int }_{\Theta }L(\theta ,a)d\pi (\theta \mid x)$, then ${\delta }_{\pi }$ is a Bayes rule.

(ii) (Admissibility.) If the Bayes rule ${\delta }_{\pi }$ is unique up to risk equivalence — any other Bayes rule has the same risk function — then ${\delta }_{\pi }$ is admissible.

Proof of (i). For any rule $\delta$, Tonelli's theorem (the integrand is non-negative) lets the Bayes risk be reordered:

$$r(\pi ,\delta )={\int }_{\Theta }{\int }_{\mathcal{X}}L(\theta ,\delta (x))d{P}_{\theta }(x)d\pi (\theta )={\int }_{\mathcal{X}}{\int }_{\Theta }L(\theta ,\delta (x))d\pi (\theta \mid x)dm(x)={\int }_{\mathcal{X}}\rho (\pi ,\delta (x)\mid x)dm(x).$$

The Bayes risk is thus the $m$-integral of the posterior expected loss evaluated at the chosen action. To minimise the integral it suffices to minimise the integrand $x$ by $x$, which is what ${\delta }_{\pi }$ does by hypothesis. Hence $r(\pi ,{\delta }_{\pi })\le r(\pi ,\delta )$ for every $\delta$, so ${\delta }_{\pi }$ attains $r(\pi )$ and is a Bayes rule. $\square$

Proof of (ii). Suppose ${\delta }_{\pi }$ is inadmissible. Then some ${\delta }^{\prime }$ dominates it: $R(\theta ,{\delta }^{\prime })\le R(\theta ,{\delta }_{\pi })$ for all $\theta$, strictly for some ${\theta }_0$. Integrating against $\pi$,

$$r(\pi ,{\delta }^{\prime })={\int }_{\Theta }R(\theta ,{\delta }^{\prime })d\pi (\theta )\le {\int }_{\Theta }R(\theta ,{\delta }_{\pi })d\pi (\theta )=r(\pi ).$$

Because $r(\pi )$ is the minimum Bayes risk, the reverse inequality $r(\pi ,{\delta }^{\prime })\ge r(\pi )$ also holds, so $r(\pi ,{\delta }^{\prime })=r(\pi )$ and ${\delta }^{\prime }$ is itself a Bayes rule for $\pi$. By the uniqueness hypothesis ${\delta }^{\prime }$ has the same risk function as ${\delta }_{\pi }$, contradicting the strict inequality at ${\theta }_0$. Hence no dominating ${\delta }^{\prime }$ exists and ${\delta }_{\pi }$ is admissible. $\square$

Bridge. This theorem builds toward the complete-class theory and the minimax constructions of the Advanced section, and appears again in the Rao-Blackwell improvement 45.01.02 and the Bayes-optimality of the Neyman-Pearson test 45.02.01. This is exactly the device that converts a global optimisation over rules into a pointwise optimisation over actions, paid for by the prior, and it generalises the elementary posterior-mean estimator of 26.07.01 to an arbitrary loss: under squared error the posterior-expected-loss minimiser is the posterior mean, under absolute error the posterior median, under 0-1 loss the posterior mode. The foundational reason Bayes rules are the workhorse is that uniqueness upgrades them to admissible rules, so the Bayes construction simultaneously delivers optimality-on-average and the no-dominance guarantee. Putting these together with the convexity of the risk set, the minimax rule turns out to be dual to a least-favourable prior — the same prior-averaging integral, now read as a saddle point — which is the connection the Advanced section makes precise via the separating-hyperplane geometry of 44.01.01.

Exercises Intermediate+

Advanced results Master

The geometry of the risk set and admissibility

Take $\Theta =\{{\theta }_1,\dots ,{\theta }_k\}$ finite and let $S\subseteq {\mathbb{R}}^k$ be the risk set of all randomized rules, a convex set by mixing 44.01.01. Three principles become three geometric operations on $S$. A rule is admissible if and only if its risk vector lies on the lower-left boundary of $S$: the set of points $s\in S$ for which no other point of $S$ is coordinatewise $\le s$ with a strict inequality, the Pareto frontier of $S$. A rule is Bayes for $\pi$ if and only if its risk vector minimises the linear functional $⟨\pi ,\cdot ⟩$ over $S$, that is, $S$ has a supporting hyperplane with outer normal $\pi$ at that point; as $\pi$ ranges over the simplex, these supporting points trace out the closed convex lower boundary. A rule is minimax if and only if its risk vector minimises ${\max }_i{s}_i$, the lowest corner of the smallest down-translate of the positive orthant meeting $S$. The first identification shows that admissible rules are exactly the exposed and supporting points of the lower boundary; the second that every Bayes rule is a supporting point and hence admissible whenever the support is strict; the third that the minimax point sits where the diagonal $s_1=\cdots =s_k$ meets the lower boundary, which is generically a Bayes point ^{[Ferguson §2]}.

The minimax theorem and least favourable priors

The minimax-Bayes duality is the statistical face of von Neumann's minimax theorem applied to the loss game between Nature (choosing $\theta$, or a prior $\pi$) and the statistician (choosing $\delta$). Under conditions guaranteeing a saddle point — compactness of the (randomized) decision space and risk-set convexity — the value of the game exists:

$$\underset{\delta }{\inf }\underset{\pi }{\sup }r(\pi ,\delta )=\underset{\pi }{\sup }\underset{\delta }{\inf }r(\pi ,\delta ),\text{i.e.}\underset{\delta }{\inf }\underset{\theta }{\sup }R(\theta ,\delta )=\underset{\pi }{\sup }r(\pi ).$$

A prior ${\pi }^{\ast }$ attaining the right side is least favourable: it is the prior under which the best achievable Bayes risk is largest, and the Bayes rule against ${\pi }^{\ast }$ is minimax ^{[Berger §5.2]}. The standard production recipe inverts this: guess an equalizer rule ${\delta }_0$ with constant risk, exhibit a prior ${\pi }^{\ast }$ for which ${\delta }_0$ is Bayes, and conclude ${\delta }_0$ is minimax — the constant-risk and Bayes conditions together forcing the saddle. When no proper least favourable prior exists, one uses a least favourable sequence ${\pi }_n$ with $r({\pi }_n)\to {\sup }_{\theta }R(\theta ,{\delta }_0)$, the device by which the sample mean $\bar{X}$ is shown minimax for the normal mean via priors $N(0,{\tau }^2)$ with $\tau \to \infty$ ^{[Lehmann & Casella §5.2]}.

Stein's phenomenon: admissibility is dimension-sensitive

The sample mean $\bar{X}$ is admissible as an estimator of a normal mean in dimensions one and two under squared-error loss, but Charles Stein proved in 1956 that for $\theta \in {\mathbb{R}}^p$ with $p\ge 3$ the estimator ${\delta }_0(X)=X$ is inadmissible ^{[Stein 1956]}: the James-Stein estimator ${\delta }_{JS}(X)=(1-(p-2)/\Vert X{\Vert }^2)X$ dominates it everywhere, shrinking toward the origin. The risk improvement is computed via Stein's unbiased risk identity ${\mathbb{E}}_{\theta }\Vert \delta (X)-\theta {\Vert }^2=p+{\mathbb{E}}_{\theta }[2\nabla \cdot g(X)+\Vert g(X){\Vert }^2]$ for $\delta (X)=X+g(X)$ ^{[Lehmann & Casella §5.5]}. The phenomenon shows admissibility is not a local property of a single coordinate: borrowing strength across coordinates strictly reduces total risk despite each coordinate's estimate becoming biased. James-Stein is itself inadmissible (its positive-part truncation dominates it), so the example also illustrates that exhibiting a dominator does not produce an admissible terminus.

Complete class theorems

A class $\mathcal{C}$ of rules is complete if for every rule $\delta \notin \mathcal{C}$ there is a ${\delta }^{\prime }\in \mathcal{C}$ dominating it, and essentially complete if the dominating ${\delta }^{\prime }$ is merely no-worse. The structural payoff is that one may restrict the search for good rules to a complete class without loss. The central results, due to Wald and refined by Stein, Le Cam, and Brown, state that under convexity of the loss and mild regularity the Bayes rules and their limits form a complete class, and conversely every admissible rule is a (generalized or limiting) Bayes rule ^{[Berger §8]}. For finite $\Theta$ the closed lower boundary of the convex risk set $S$ is the minimal complete class, and its supporting hyperplanes are exactly the priors, so "admissible," "supporting point of $S$," and "Bayes or limit of Bayes" coincide. The complete-class theorems are the converse direction of the admissibility-of-Bayes-rules result: not only is every (unique) Bayes rule admissible, but every admissible rule is, in the limit, Bayes.

Synthesis. The central insight of decision theory is that risk turns the choice of a statistical procedure into a partial order — dominance — whose maximal elements (admissible rules) are exactly the supporting points of the convex risk set, and the three classical principles are three ways of selecting among them. The foundational reason Bayes rules dominate the theory is the posterior-expected-loss characterization: it makes the Bayes rule computable pointwise, makes unique Bayes rules admissible, and — through the complete-class theorems — makes every admissible rule a limit of Bayes rules, so the Bayes construction generalises from one optimality principle into the generating mechanism for the whole admissible frontier. Putting these together, the minimax rule is dual to a least favourable prior in the literal saddle-point sense, the minimax value equalling ${\sup }_{\pi }r(\pi )$, which is exactly von Neumann's game value read on the loss matrix and supported by the separating-hyperplane geometry of 44.01.01. The Stein phenomenon shows the order is genuinely global and dimension-sensitive, and the entire apparatus appears again in estimation 45.01.02, testing 45.02.01, and the Bayesian inference of 26.07.01, where these same objects — loss, risk, prior, posterior — recur as the unifying language. The bridge is the convex risk set: feasibility, optimality, admissibility, the Bayes-minimax duality, and the complete-class theorems are one theory about supporting hyperplanes of a single convex object.

Full proof set Master

Proposition 1 (the risk set is convex; Bayes points support it). For finite $\Theta =\{{\theta }_1,\dots ,{\theta }_k\}$, the risk set $S=\{R(\delta ):=(R({\theta }_i,\delta ){)}_i:\delta \text{ randomized}\}$ is convex, and $\delta$ is Bayes for $\pi$ in the simplex if and only if $⟨\pi ,R(\delta )⟩={\min }_{s\in S}⟨\pi ,s⟩$.

Proof. Given rules ${\delta }^{\prime },{\delta }^{\prime \prime }$ and $\alpha \in [0,1]$, the randomized rule that uses ${\delta }^{\prime }$ with probability $\alpha$ and ${\delta }^{\prime \prime }$ with probability $1-\alpha$ (an independent coin flip before seeing data) has risk $R({\theta }_i,\cdot )=\alpha R({\theta }_i,{\delta }^{\prime })+(1-\alpha )R({\theta }_i,{\delta }^{\prime \prime })$ at each ${\theta }_i$ by linearity of expectation, so $\alpha R({\delta }^{\prime })+(1-\alpha )R({\delta }^{\prime \prime })=R(\text{mixture})\in S$; hence $S$ is convex (the mixing operation of 44.01.01). For the Bayes characterization, $r(\pi ,\delta )={\sum }_i{\pi }_{i}R({\theta }_i,\delta )=⟨\pi ,R(\delta )⟩$, so minimising Bayes risk over $\delta$ is minimising the linear functional $⟨\pi ,\cdot ⟩$ over $S$. The minimiser is a point at which the hyperplane $\{s:⟨\pi ,s⟩=r(\pi )\}$ supports $S$ from below. $\square$

Proposition 2 (a Bayes rule with constant risk is minimax). If ${\delta }_0$ is Bayes for some prior $\pi$ and $R(\theta ,{\delta }_0)=c$ is constant in $\theta$, then ${\delta }_0$ is minimax and $\pi$ is least favourable.

Proof. For any rule $\delta$, since ${\delta }_0$ is Bayes, $r(\pi ,{\delta }_0)\le r(\pi ,\delta )$. The left side is $\int R(\theta ,{\delta }_0)d\pi =c$. The right side is a prior average and so cannot exceed the maximum risk: $r(\pi ,\delta )=\int R(\theta ,\delta )d\pi \le {\sup }_{\theta }R(\theta ,\delta )$. Chaining, $c\le {\sup }_{\theta }R(\theta ,\delta )$ for every $\delta$. Taking the infimum over $\delta$, $c\le {\inf }_{\delta }{\sup }_{\theta }R(\theta ,\delta )=\bar{V}$. But ${\delta }_0$ itself has ${\sup }_{\theta }R(\theta ,{\delta }_0)=c$, so $\bar{V}\le c$. Hence $\bar{V}=c={\sup }_{\theta }R(\theta ,{\delta }_0)$ and ${\delta }_0$ is minimax. Moreover ${\sup }_{{\pi }^{\prime }}r({\pi }^{\prime })\ge r(\pi )=c=\bar{V}\ge {\sup }_{{\pi }^{\prime }}r({\pi }^{\prime })$ (the last step is weak duality $r({\pi }^{\prime })\le \bar{V}$ for every prior), so $\pi$ attains ${\sup }_{{\pi }^{\prime }}r({\pi }^{\prime })$ and is least favourable. $\square$

Proposition 3 (weak duality: every Bayes value is below the minimax value). For every prior $\pi$, $r(\pi )\le \bar{V}={\inf }_{\delta }{\sup }_{\theta }R(\theta ,\delta )$.

Proof. For any fixed $\delta$, $r(\pi ,\delta )=\int R(\theta ,\delta )d\pi \le {\sup }_{\theta }R(\theta ,\delta )$ because an average over $\pi$ is dominated by the supremum. Taking the infimum over $\delta$ on both sides, $r(\pi )={\inf }_{\delta }r(\pi ,\delta )\le {\inf }_{\delta }{\sup }_{\theta }R(\theta ,\delta )=\bar{V}$. (This is the max-min inequality ${\sup }_{\pi }{\inf }_{\delta }r\le {\inf }_{\delta }{\sup }_{\theta }R$ specialized; equality is the minimax theorem, the saddle realized by a least favourable ${\pi }^{\ast }$ and its Bayes rule.) $\square$

Proposition 4 (admissibility of a unique Bayes rule). If ${\delta }_{\pi }$ is Bayes for $\pi$ with $r(\pi )<\infty$ and every Bayes rule for $\pi$ has the same risk function as ${\delta }_{\pi }$, then ${\delta }_{\pi }$ is admissible.

Proof. This is part (ii) of the Key theorem, reproduced for the proof set. If ${\delta }^{\prime }$ dominated ${\delta }_{\pi }$, integrating the dominance inequality $R(\theta ,{\delta }^{\prime })\le R(\theta ,{\delta }_{\pi })$ against $\pi$ gives $r(\pi ,{\delta }^{\prime })\le r(\pi )$, hence $r(\pi ,{\delta }^{\prime })=r(\pi )$ since $r(\pi )$ is minimal, so ${\delta }^{\prime }$ is also Bayes for $\pi$. By the uniqueness hypothesis $R(\cdot ,{\delta }^{\prime })=R(\cdot ,{\delta }_{\pi })$, contradicting the strict inequality that domination requires at some ${\theta }_0$. So no dominator exists. $\square$

Connections Master

• Unbiased estimation and the Rao-Blackwell / Lehmann-Scheffé theorems 45.01.02 are the squared-error specialization of this framework: conditioning an estimator on a sufficient statistic is a convexity-of-loss argument that lowers risk, the same risk and dominance order defined here, and the minimum-variance-unbiased estimator is the admissible representative inside the unbiased class.

• Hypothesis testing and the Neyman-Pearson lemma 45.02.01 are the two-action decision problem under 0-1 (or weighted 0-1) loss; the most powerful test is the Bayes rule of Exercise 5 for the corresponding prior, and the admissibility and complete-class structure of tests is the same risk-set geometry, now over the power/size plane.

• Bayesian inference 26.07.01 supplies the prior, posterior, and posterior-expected-loss machinery that the Key theorem turns into the Bayes rule; this unit is the decision-theoretic completion of that survey, replacing "report the posterior" with "choose the action minimising posterior expected loss for a stated loss function."

• The convex-set and supporting-hyperplane calculus of 44.01.01 is the geometric engine: the risk set is convex by rule-mixing, Bayes rules are its supporting points, admissible rules are its lower boundary, and the minimax-Bayes duality is the separating-hyperplane / saddle-point structure read on this single convex object, exactly as Lagrangian duality reads it on the value set.

Historical & philosophical context Master

The decision-theoretic reframing of statistics is due to Abraham Wald, whose Statistical Decision Functions ^{[Wald 1950]} (Wiley, 1950) introduced the loss function, the risk function, randomized decision rules, and the complete-class program, casting estimation and testing as special cases of a single game against Nature; the minimax principle and the proof that Bayes and limiting-Bayes rules form a complete class are Wald's. The game-theoretic substrate traces to John von Neumann's 1928 minimax theorem for zero-sum games, which supplies the saddle-point existence behind the minimax-Bayes duality, and the decision-game reading was sharpened by Wald and by L. J. Savage's subjective-expected-utility foundations in The Foundations of Statistics (1954).

Charles Stein's 1956 Berkeley Symposium paper ^{[Stein 1956]} (Proc. Third Berkeley Symposium 1, 197-206) proved the inadmissibility of the sample mean in three or more dimensions, a result so counter to intuition that it reoriented the field's understanding of admissibility; the explicit dominating estimator was given by James and Stein in 1961. The systematic theory of admissibility and complete classes was developed through the 1950s-1970s by Stein, Lucien Le Cam, and Lawrence Brown, whose differential-equation characterization of admissibility for the normal mean closed the dimension-one and -two admissibility questions. Thomas Ferguson's Mathematical Statistics: A Decision Theoretic Approach ^{[Ferguson §2]} (1967) gave the geometric risk-set exposition, and James Berger's Statistical Decision Theory and Bayesian Analysis ^{[Berger §4-§5]} (1985) and Lehmann and Casella's Theory of Point Estimation ^{[Lehmann & Casella §5]} (1998) are the standard modern references, the latter integrating the decision framework with the classical optimality theory of estimation.

Bibliography Master

@book{wald1950statistical,
  author    = {Wald, Abraham},
  title     = {Statistical Decision Functions},
  publisher = {Wiley},
  year      = {1950}
}

@inproceedings{stein1956inadmissibility,
  author    = {Stein, Charles},
  title     = {Inadmissibility of the usual estimator for the mean of a multivariate normal distribution},
  booktitle = {Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability},
  volume    = {1},
  pages     = {197--206},
  publisher = {University of California Press},
  year      = {1956}
}

@book{ferguson1967mathematical,
  author    = {Ferguson, Thomas S.},
  title     = {Mathematical Statistics: A Decision Theoretic Approach},
  publisher = {Academic Press},
  year      = {1967}
}

@book{berger1985statistical,
  author    = {Berger, James O.},
  title     = {Statistical Decision Theory and Bayesian Analysis},
  edition   = {2nd},
  series    = {Springer Series in Statistics},
  publisher = {Springer},
  year      = {1985}
}

@book{lehmanncasella1998theory,
  author    = {Lehmann, Erich L. and Casella, George},
  title     = {Theory of Point Estimation},
  edition   = {2nd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {1998}
}

@book{casellaberger2002statistical,
  author    = {Casella, George and Berger, Roger L.},
  title     = {Statistical Inference},
  edition   = {2nd},
  publisher = {Duxbury},
  year      = {2002}
}