Bayes Estimation Under Loss: Posterior Mean, Median, and Mode

Intuition Beginner

You have done a Bayesian analysis and you hold a posterior — a full probability distribution for the unknown quantity, given your data and your prior belief. Often, though, someone wants a single number from you. What is your one best guess for the rate, the mean, the proportion? A whole distribution is honest, but a meeting needs an answer. The question of this unit is: which single number should you report?

The answer is not "whatever feels central." It depends on how you will be penalised for being wrong. If a miss of two units is exactly four times as painful as a miss of one unit, you are being scored by squared error, and the best single number is the posterior mean. If every unit of miss costs the same, no matter the direction, you are being scored by absolute error, and the best number is the posterior median. If the only thing that matters is being exactly right — any miss at all is one full unit of pain — then you should report the posterior mode, the single most probable value.

So the same posterior yields three different "best guesses," and the loss function picks which one. This is the heart of Bayesian point estimation: first build the posterior, then choose the summary that the cost of mistakes demands. Reporting a number without saying which loss you assumed is like quoting a price without a currency.

One more idea runs through everything here. Because the posterior blends prior and data, the Bayesian estimate sits between what you believed beforehand and what the data alone would say. It is pulled — shrunk — toward the prior. When data are scarce, the prior holds more sway; when data are abundant, the data win. That gentle shrinkage is not a bug. It is the same instinct that powers ridge regression and many modern estimators.

Visual Beginner

Figure: a single skewed posterior density drawn over a horizontal axis for the unknown $\theta$. Three vertical markers fall at different places. The tallest point of the curve is the mode (the peak). To its right sits the median (the value with half the area on each side). Farther right still sits the mean (the balance point of the area, dragged outward by the long right tail). For a symmetric bell the three would coincide; the skew is what separates them, and each is the right answer under a different penalty for error.

 posterior
 density
   |            .-.            mode  = peak of the curve
   |           /   \           median = splits area 50/50
   |          /     \          mean  = balance point (pulled right by the tail)
   |         /        \____
   |        /              \________
   +--------|----|----|-------------------> theta
          mode median mean
        (0-1 loss)(abs err)(squared err)
   symmetric posterior: all three coincide
   skewed posterior:    mode < median < mean (for a right skew)

Worked example Beginner

A website ran a small test: out of $10$ visitors, $8$ clicked a button. You want one number for the click rate $p$. Before the test you were fairly open-minded, so you used a uniform prior on $p$, which is the same as $\text{Beta}(1,1)$. For the beta-binomial pair, the posterior after $8$ successes and $2$ failures is $\text{Beta}(1+8,1+2)=\text{Beta}(9,3)$.

Step 1. Posterior mean (squared-error loss). For a $\text{Beta}(a,b)$ distribution the mean is $a/(a+b)$. Here that is $9/(9+3)=9/12=0.75$.

Step 2. Posterior mode (0-1 loss). For $\text{Beta}(a,b)$ with $a,b>1$ the mode is $(a-1)/(a+b-2)$. Here that is $(9-1)/(12-2)=8/10=0.80$. Notice this equals the raw sample proportion $8/10$ — the mode of a flat-prior posterior is the maximum-likelihood guess.

Step 3. Posterior median (absolute-error loss). The median of $\text{Beta}(9,3)$ has no tidy formula, but a good approximation for $a,b>1$ is $(a-1/3)/(a+b-2/3)$, giving $(9-0.333)/(12-0.667)=8.667/11.333\approx 0.765$.

Step 4. Compare. The three summaries are mode $0.80$, median $0.765$, mean $0.75$. They are ordered mode $>$ median $>$ mean because this posterior is skewed to the left (a long tail toward small $p$), which drags the mean down below the peak.

What this tells us. One posterior, three legitimate "best guesses," and the gap between them ($0.75$ versus $0.80$) is the size of the decision you make when you choose a loss function. With more data the posterior tightens and the three numbers march together; with this little data, the choice of loss visibly matters.

Check your understanding Beginner

Formal definition Intermediate+

The setting is the decision problem of 45.01.01 specialised to point estimation. A parameter $\theta$ ranges over $\Theta \subseteq {\mathbb{R}}^d$ with prior probability measure $\pi$; data $X$ has sampling law $P_{\theta }$ with density $f(x\mid \theta )$ against a dominating measure. The action space is $\mathcal{A}=\Theta$ and a loss $L:\Theta \times \mathcal{A}\to [0,\infty )$ scores an estimate. The joint law of $(\theta ,X)$ disintegrates as

$$d\pi (\theta )f(x\mid \theta )d\mu (x)=dm(x)d\pi (\theta \mid x),$$

where $m(x)={\int }_{\Theta }f(x\mid \theta )d\pi (\theta )$ is the marginal and $\pi (\cdot \mid x)$ is the posterior, the regular conditional law of $\theta$ given $X=x$ (the conditioning operation of 37.04.01, surveyed elementarily in 26.07.01).

Definition (posterior expected loss). For an action $a\in \mathcal{A}$, the posterior expected loss at $x$ is

$$\rho (\pi ,a\mid x)={\int }_{\Theta }L(\theta ,a)d\pi (\theta \mid x)=\mathbb{E}\left[L(\theta ,a)\mid X=x\right].$$

Definition (Bayes estimator). A Bayes estimator ${\delta }_{\pi }$ is any rule that minimises the Bayes risk $r(\pi ,\delta )={\int }_{\Theta }R(\theta ,\delta )d\pi (\theta )$ of 45.01.01. Equivalently — by the characterisation theorem below — ${\delta }_{\pi }(x)$ minimises the posterior expected loss $a\mapsto \rho (\pi ,a\mid x)$ for $m$-almost every $x$.

The three canonical losses and their minimisers:

• Squared-error loss $L(\theta ,a)=\Vert \theta -a{\Vert }^2$. The Bayes estimator is the posterior mean ${\delta }_{\pi }(x)=\mathbb{E}[\theta \mid x]$.
• Absolute-error loss $L(\theta ,a)=|\theta -a|$ (scalar $\theta$). The Bayes estimator is a posterior median of $\pi (\cdot \mid x)$.
• $0$-$K$ loss $L_K(\theta ,a)=1\{|\theta -a|>K\}$. As $K\downarrow 0$ the minimiser tends to the posterior mode $\arg {\max }_{\theta }\pi (\theta \mid x)$, the maximum a posteriori (MAP) estimate. On a finite or discrete $\Theta$ the loss $L(\theta ,a)=1\{a\ne \theta \}$ is already the posterior mode exactly.

Definition (generalized Bayes estimator). When $\pi$ is an improper prior — a $\sigma$-finite measure with $\pi (\Theta )=\infty$, such as Lebesgue measure on $\mathbb{R}$ or $1/{\sigma }^2$ on the variance — the Bayes risk is undefined, but if the formal posterior $\pi (\theta \mid x)\propto f(x\mid \theta )d\pi (\theta )$ is a proper probability measure for $m$-almost every $x$, the rule minimising $\rho (\pi ,a\mid x)$ pointwise is a generalized Bayes estimator. Generalized Bayes estimators are the bridge to many classical frequentist estimators: under the flat prior on a normal mean the generalized Bayes estimator for squared error is the sample mean.

Counterexamples to common slips

• The MAP estimate is not coordinate-free. The mode of a density depends on the parametrisation: reparametrising $\theta \mapsto g(\theta )$ multiplies the density by a Jacobian, so the MAP of $\theta$ and of $g(\theta )$ need not satisfy $g({\hat{\theta }}_{\text{MAP}})={\widehat{g(\theta )}}_{\text{MAP}}$. The posterior mean and median are likewise not equivariant under nonlinear $g$, but the mode's failure is the one most often mistaken for a maximum-likelihood-style invariance.
• A flat prior is not "no information." The improper uniform prior is not invariant under reparametrisation and can yield an improper posterior (no valid Bayes estimator at all); flatness is a choice with consequences, not the absence of one.
• The posterior median can be a set. When the posterior has an atom or a flat stretch at probability one-half, the absolute-error minimiser is an interval of medians; any point of it is a Bayes estimator, so the Bayes estimator need not be unique even though its Bayes risk is.

Key theorem with proof Intermediate+

The organising result is that the global problem of minimising Bayes risk over all estimators reduces to a pointwise minimisation over actions, and that for the three canonical losses this pointwise problem has the mean, median, and mode as explicit solutions.

Theorem (Bayes estimators as posterior-loss minimisers). Fix a prior $\pi$ with finite Bayes value $r(\pi )<\infty$, and suppose the joint law disintegrates as above with marginal $m$.

(i) (Reduction.) If for $m$-almost every $x$ the action ${\delta }_{\pi }(x)$ minimises $a\mapsto \rho (\pi ,a\mid x)$, then ${\delta }_{\pi }$ is a Bayes estimator.

(ii) (Squared error.) For $L(\theta ,a)=\Vert \theta -a{\Vert }^2$ the unique minimiser is the posterior mean ${\delta }_{\pi }(x)=\mathbb{E}[\theta \mid x]$.

(iii) (Absolute error.) For scalar $\theta$ and $L(\theta ,a)=|\theta -a|$, any posterior median minimises $\rho$.

(iv) (0-1 / mode.) For $L(\theta ,a)=1\{a\ne \theta \}$ on a discrete $\Theta$ the minimiser is the posterior mode; for $L_K(\theta ,a)=1\{|\theta -a|>K\}$ on a continuum the minimiser tends to a mode of a continuous bounded posterior density as $K\downarrow 0$.

Proof of (i). For any estimator $\delta$, the loss integrand is non-negative, so Tonelli's theorem permits reordering the Bayes risk along the disintegration:

$$r(\pi ,\delta )={\int }_{\Theta }{\int }_{\mathcal{X}}L(\theta ,\delta (x))d{P}_{\theta }(x)d\pi (\theta )={\int }_{\mathcal{X}}{\int }_{\Theta }L(\theta ,\delta (x))d\pi (\theta \mid x)dm(x)={\int }_{\mathcal{X}}\rho (\pi ,\delta (x)\mid x)dm(x).$$

The Bayes risk is the $m$-integral of the posterior expected loss at the chosen action. An integral of non-negative terms is minimised by minimising the integrand for each $x$, which is what ${\delta }_{\pi }$ does. Hence $r(\pi ,{\delta }_{\pi })\le r(\pi ,\delta )$ for all $\delta$. $\square$

Proof of (ii). Fix $x$ and write $\mu =\mathbb{E}[\theta \mid x]$, assumed finite since $r(\pi )<\infty$. For any action $a$,

$$\rho (\pi ,a\mid x)=\mathbb{E}\left[\Vert \theta -a{\Vert }^2\mid x\right]=\mathbb{E}\left[\Vert \theta -\mu {\Vert }^2\mid x\right]+\Vert \mu -a{\Vert }^2,$$

because the cross term $2\mathbb{E}[(\theta -\mu )\mid x{]}^{\top }(\mu -a)=0$ by the definition of $\mu$ as the conditional mean (the orthogonality of the conditional-expectation projection, 37.04.01). The first term is free of $a$ and the second is minimised uniquely at $a=\mu$. Hence the posterior mean is the unique squared-error Bayes estimator. $\square$

Proof of (iii). Fix $x$ and abbreviate the posterior $F(t)=\pi (\theta \le t\mid x)$. The posterior expected loss is

$$\rho (a)={\int }_{-\infty }^a(a-\theta )d\pi (\theta \mid x)+{\int }_a^{\infty }(\theta -a)d\pi (\theta \mid x).$$

For $a<b$, a direct computation gives

$$\rho (b)-\rho (a)={\int }_a^b[\pi (\theta \le t\mid x)-\pi (\theta >t\mid x)]dt={\int }_a^b[2F(t)-1]dt,$$

using ${\rho }^{\prime }(a)=F(a^{-})+F(a)-1$ where $F$ is differentiable and the integrated form otherwise. The integrand $2F(t)-1$ is $\le 0$ for $t$ below a median and $\ge 0$ above it, so $\rho$ decreases up to a median and increases after; any $m$ with $F(m^{-})\le \frac{1}{2}\le F(m)$ is a global minimiser. Convexity of $|\cdot |$ guarantees the stationary condition is sufficient. So a posterior median minimises $\rho$. $\square$

Proof of (iv). On discrete $\Theta$, $\rho (\pi ,a\mid x)={\sum }_{\theta \ne a}\pi (\theta \mid x)=1-\pi (a\mid x)$, so minimising $\rho$ is maximising $\pi (a\mid x)$, the posterior mode. For the continuous $0$-$K$ loss, $\rho (\pi ,a\mid x)=1-\pi (|\theta -a|\le K\mid x)=1-{\int }_{a-K}^{a+K}p(\theta \mid x)d\theta$ where $p$ is the posterior density; minimising $\rho$ maximises the posterior mass in the band $[a-K,a+K]$. As $K\downarrow 0$, dividing the mass by $2K$ and passing to the limit, the maximising centre converges to a maximiser of the continuous bounded density $p(\cdot \mid x)$, a posterior mode. $\square$

Bridge. This theorem builds toward the conjugate closed forms and the admissibility theory of the Advanced section, and it appears again in ridge regression as a posterior mean 45.06.03 and James-Stein shrinkage as empirical Bayes 45.06.04. This is exactly the device that converts a global optimisation over estimators into a pointwise optimisation over actions, paid for by the prior, and it generalises the single-loss Bayes-rule statement of 45.01.01 into a menu — mean, median, mode — indexed by the loss. The central insight is that the posterior is a sufficient summary for every loss at once: build it once, then read off whatever summary the cost of error demands, so that the choice of point estimate is downstream of, and separable from, the inference itself. Putting these together, the squared-error case identifies the Bayes estimator with the conditional-expectation projection of 37.04.01, which is why the posterior mean inherits the orthogonality, tower, and variance-decomposition properties of conditional expectation, and why the normal-normal Bayes estimator turns out to be dual to the ridge penalty when the prior precision is read as a regularisation weight.

Exercises Intermediate+

Advanced results Master

Conjugacy and the algebra of shrinkage

For an exponential-family likelihood $f(x\mid \theta )=h(x)\exp (\eta (\theta {)}^{\top }T(x)-A(\theta ))$ a conjugate prior of the form $\pi (\theta )\propto \exp (\eta (\theta {)}^{\top }\nu -\kappa A(\theta ))$ yields a posterior in the same family with updated hyperparameters $\nu \mapsto \nu +T(x)$, $\kappa \mapsto \kappa +1$. The three worked families instantiate one mechanism. The normal-normal posterior mean ${\mu }_n=({\sigma }^{-2}x+{\tau }^{-2}{\mu }_0)/({\sigma }^{-2}+{\tau }^{-2})$ is a precision-weighted average; the beta-binomial mean $(\alpha +x)/(\alpha +\beta +n)$ adds data counts to pseudo-counts; the gamma-Poisson mean $(\alpha +S)/(\beta +n)$ adds the sufficient statistic to the prior rate. In every case the Bayes estimator under squared error is a convex combination of a prior centre and a data summary, the data weight rising to one as information accumulates ^{[Robert Ch. 3]}. The precision-weighting view exposes the shrinkage interpretation that the next chapter formalises: the prior precision is a regularisation strength, and the normal-normal estimator is, term for term, the ridge solution of 45.06.03 with penalty $\lambda ={\sigma }^2/{\tau }^2$. The full exponential-family conjugacy theory — including the characterisation of which families admit conjugate priors and the linearity of the posterior expectation of the mean parameter in $T(x)$ (Diaconis-Ylvisaker) — is the content of 45.03.02.

Admissibility of proper Bayes estimators

A proper Bayes estimator with finite Bayes risk is admissible whenever it is essentially the unique minimiser, by the integration argument of the Key theorem; for squared-error loss with a proper prior of full support and continuous risk functions, admissibility holds without an explicit uniqueness hypothesis, since a strict local risk improvement spreads to a positive-prior-mass neighbourhood and would strictly lower the Bayes risk ^{[Berger §8.1]}. The converse direction — that admissible estimators are limits of Bayes estimators — is the complete-class theory of 45.01.01. The boundary cases are governed by Blyth's method and by Brown's differential-equation characterisation: an estimator is admissible if it is a limit of Bayes estimators along priors ${\pi }_n$ whose Bayes-risk gap to the candidate tends to zero. This is the route by which the generalized Bayes estimator $X$ for the normal mean is proved admissible in dimensions one and two — through a least-favourable sequence $N(0,{\tau }^2)$ with $\tau \to \infty$ — and by which Stein's inadmissibility in dimension three or more is, dually, the failure of that sequence to close the gap ^{[Lehmann & Casella §5.2]}.

Generalized Bayes, impropriety, and the frequentist bridge

Improper priors enlarge the class of Bayes estimators to recover classical frequentist procedures: the flat-prior generalized Bayes estimator of a location is the sample mean, the $1/{\sigma }^2$ prior reproduces the usual variance estimator's posterior, and Jeffreys priors $\pi (\theta )\propto \sqrt{\det I(\theta )}$ furnish reparametrisation-invariant default rules. The price is real. An improper prior can produce an improper posterior, in which case no estimator exists; marginalisation paradoxes (Dawid-Stone-Zidek) show that improper priors can yield mutually inconsistent conditional inferences; and a generalized Bayes estimator need not be admissible, the James-Stein dominance being precisely the statement that the generalized Bayes estimator $X$ is inadmissible for $p\ge 3$ under total squared-error loss ^{[Berger §4.4]}. The Bayesian-frequentist contrast crystallises at the point estimate. The frequentist evaluates an estimator by its sampling risk $R(\theta ,\delta )$ as a function of the fixed unknown $\theta$ and seeks uniform or minimax control; the Bayesian averages that risk against a prior and minimises, obtaining a single estimator computable from the posterior alone. The Bernstein-von Mises theorem of 26.07.01 reconciles them asymptotically — the posterior concentrates at the MLE with inverse-Fisher-information spread, so the squared-error Bayes estimator and the MLE agree to first order — but in finite samples the prior's shrinkage is exactly the gap, and it is this gap that ridge and James-Stein exploit on purpose.

Synthesis. The central insight is that the posterior is a loss-free sufficient summary: it is computed once from prior and data, and this is exactly the object from which every point estimate is read off, the loss function entering only at the final pointwise minimisation that selects mean, median, or mode. The foundational reason the three summaries arise from one posterior is the reduction theorem — Bayes risk is the marginal integral of posterior expected loss, so global optimality generalises into pointwise optimality and the loss merely re-weights a fixed measure. Putting these together, conjugacy makes the squared-error estimator a precision-weighted average, which is dual to the ridge penalty of 45.06.03 under the dictionary prior-precision $=$ regularisation-strength, while the multivariate sharpening of that shrinkage appears again in the James-Stein estimator of 45.06.04, whose dominance of the sample mean is the Stein inadmissibility of 45.01.01 read as empirical Bayes. The bridge is the posterior-expected-loss functional: admissibility of proper Bayes estimators, the generalized-Bayes recovery of classical rules, the conjugate shrinkage algebra, and the Bayesian-frequentist asymptotic agreement are one theory about minimising $\int L(\theta ,a)d\pi (\theta \mid x)$, with the prior's properness the single dial that separates a guaranteed-admissible estimator from a classical frequentist one.

Full proof set Master

Proposition 1 (reduction to posterior expected loss). For a prior $\pi$ with $r(\pi )<\infty$ and the disintegration $d\pi (\theta )d{P}_{\theta }(x)=dm(x)d\pi (\theta \mid x)$, ${\delta }_{\pi }$ is a Bayes estimator if and only if ${\delta }_{\pi }(x)$ minimises $a\mapsto \rho (\pi ,a\mid x)$ for $m$-almost every $x$.

Proof. By Tonelli (non-negative integrand), $r(\pi ,\delta )={\int }_{\mathcal{X}}\rho (\pi ,\delta (x)\mid x)dm(x)$ for every $\delta$, as in the Key theorem. If ${\delta }_{\pi }$ minimises the integrand $m$-a.e., it minimises the integral, so it is Bayes. Conversely, suppose ${\delta }_{\pi }$ is Bayes but on a set $B$ with $m(B)>0$ there is an action achieving strictly smaller posterior expected loss; replacing ${\delta }_{\pi }$ by that action on $B$ (a measurable selection exists by the measurable maximum theorem when $\mathcal{A}$ is Polish and $\rho$ is jointly measurable, lower-semicontinuous in $a$) strictly lowers $r(\pi ,\cdot )$, contradicting optimality. Hence the pointwise minimisation holds $m$-a.e. $\square$

Proposition 2 (posterior mean for squared error; minimal loss is posterior variance). For $L(\theta ,a)=\Vert \theta -a{\Vert }^2$ with finite posterior second moment, the unique minimiser of $\rho (\pi ,\cdot \mid x)$ is $\mathbb{E}[\theta \mid x]$, and ${\min }_a\rho =\mathrm{tr}\mathrm{Cov}(\theta \mid x)$.

Proof. Let $\mu =\mathbb{E}[\theta \mid x]$. For any $a$, $\Vert \theta -a{\Vert }^2=\Vert \theta -\mu {\Vert }^2+2(\theta -\mu {)}^{\top }(\mu -a)+\Vert \mu -a{\Vert }^2$. Taking conditional expectation, the middle term vanishes because $\mathbb{E}[\theta -\mu \mid x]=0$, leaving $\rho (\pi ,a\mid x)=\mathbb{E}[\Vert \theta -\mu {\Vert }^2\mid x]+\Vert \mu -a{\Vert }^2$. The first summand is $\mathrm{tr}\mathrm{Cov}(\theta \mid x)$, independent of $a$; the second is non-negative and zero only at $a=\mu$. So $\mu$ is the unique minimiser and the minimal value is the posterior total variance. $\square$

Proposition 3 (posterior median for absolute error). For scalar $\theta$ and $L(\theta ,a)=|\theta -a|$ with finite posterior first moment, every posterior median minimises $\rho (\pi ,\cdot \mid x)$, and the minimisers form a closed interval.

Proof. With $F(t)=\pi (\theta \le t\mid x)$, for $a<b$ Fubini gives

$$\rho (b)-\rho (a)={\int }_a^b(2F(t)-1)dt.$$

The integrand is non-decreasing in $t$ (as $F$ is non-decreasing), negative below any median and positive above, so $\rho$ is convex with derivative changing sign exactly at the median set $\{m:F(m^{-})\le \frac{1}{2}\le F(m)\}$. A convex function on $\mathbb{R}$ attains its minimum on the closed interval where its derivative vanishes or changes sign; that interval is the median set. Hence any posterior median is a minimiser and the minimiser set is a closed interval (a single point when $F$ crosses $\frac{1}{2}$ strictly). $\square$

Proposition 4 (posterior mode as 0-1 / limiting 0-K minimiser). On discrete $\Theta$ with $L(\theta ,a)=1\{a\ne \theta \}$, the minimiser of $\rho$ is the posterior mode. For a posterior with bounded continuous density $p(\cdot \mid x)$ and $L_K(\theta ,a)=1\{|\theta -a|>K\}$, any sequence of $L_K$-minimisers $a_K$ accumulates, as $K\downarrow 0$, at a maximiser of $p(\cdot \mid x)$.

Proof. Discrete case: $\rho (a)={\sum }_{\theta \ne a}\pi (\theta \mid x)=1-\pi (a\mid x)$, minimised by maximising $\pi (a\mid x)$. Continuous case: ${\rho }_K(a)=1-{\int }_{a-K}^{a+K}p(\theta \mid x)d\theta$, so the $L_K$-minimiser maximises $g_K(a)={\int }_{a-K}^{a+K}p(\theta \mid x)d\theta$. By the mean-value form, $g_K(a)/(2K)\to p(a\mid x)$ uniformly on compacts as $K\downarrow 0$ since $p$ is continuous. Let $a_K$ maximise $g_K$ and let $a^{\text{*}}$ be a subsequential limit; for any fixed $a$, $g_K(a_K)\ge g_K(a)$, and dividing by $2K$ and passing to the limit yields $p(a^{\text{*}}\mid x)\ge p(a\mid x)$. So $a^{\text{*}}$ is a mode. $\square$

Proposition 5 (admissibility of a unique proper Bayes estimator). If ${\delta }_{\pi }$ is Bayes for a proper prior $\pi$ with $r(\pi )<\infty$ and every Bayes estimator for $\pi$ shares its risk function, then ${\delta }_{\pi }$ is admissible.

Proof. If ${\delta }^{\prime }$ dominated ${\delta }_{\pi }$, integrating $R(\theta ,{\delta }^{\prime })\le R(\theta ,{\delta }_{\pi })$ against $\pi$ gives $r(\pi ,{\delta }^{\prime })\le r(\pi )$, hence equality by minimality, so ${\delta }^{\prime }$ is Bayes for $\pi$. By hypothesis $R(\cdot ,{\delta }^{\prime })=R(\cdot ,{\delta }_{\pi })$, contradicting the strict inequality domination requires at some ${\theta }_0$. No dominator exists. For squared error the uniqueness hypothesis is discharged by Proposition 2, which makes the minimiser unique $m$-a.e. $\square$

Connections Master

• Statistical decision theory 45.01.01 supplies the loss-risk-Bayes-risk scaffold this unit specialises: the abstract Bayes rule of that unit becomes here the concrete posterior mean, median, and mode, and the admissibility-of-unique-Bayes-rules theorem is the same integration argument, now applied to point estimation under the three canonical losses.

• The elementary Bayesian survey 26.07.01 introduces the prior, posterior, conjugate families, and the bare assertion that the posterior mean/median/mode minimise the three losses; this unit is the decision-theoretic completion that proves those assertions and embeds them in the Bayes-risk reduction, replacing "report the posterior" with "minimise posterior expected loss for a stated loss."

• Conditional expectation 37.04.01 is the analytic substrate: the posterior mean is the conditional expectation $\mathbb{E}[\theta \mid X]$, so its optimality under squared error is the orthogonal-projection property of conditional expectation, and the tower property and conditional-variance decomposition transfer directly to Bayes-risk computations.

• Ridge regression and regularised estimation 45.06.03 are the posterior mean under a Gaussian prior on coefficients: the precision-weighted shrinkage of the normal-normal Bayes estimator computed here is the ridge penalty with $\lambda ={\sigma }^2/{\tau }^2$, so ridge is Bayes estimation under squared-error loss with an explicit conjugate prior.

• The James-Stein estimator 45.06.04 is the multivariate, empirical-Bayes sharpening of this shrinkage: it estimates the prior variance from the data and shrinks coordinatewise, and its dominance of the sample mean for $p\ge 3$ is the Stein inadmissibility of 45.01.01 read through the generalized-Bayes lens of this unit.

Historical & philosophical context Master

The reduction of Bayesian estimation to minimising posterior expected loss is implicit in Laplace's eighteenth-century work on inverse probability and explicit in the decision-theoretic reframing of Abraham Wald, whose Statistical Decision Functions ^{[Wald 1950]} (Wiley, 1950) made loss, risk, and Bayes solutions the primitive objects. David Blackwell and Meyer Girshick's Theory of Games and Statistical Decisions ^{[Blackwell & Girshick 1954]} (Wiley, 1954) gave the posterior-risk computation of Bayes solutions in the form used here, and Leonard J. Savage's The Foundations of Statistics (1954) supplied the subjective-expected-utility axioms under which minimising posterior expected loss is the unique coherent decision rule. The identification of the squared-error minimiser with the posterior mean, the absolute-error minimiser with the median, and the 0-1 minimiser with the mode is standard by the time of Raiffa and Schlaifer's Applied Statistical Decision Theory (1961), which also systematised the conjugate-family closed forms.

The admissibility theory of proper Bayes estimators was developed by Charles Stein, Lawrence Brown, and others through the 1950s-1970s; Brown's differential-equation characterisation of admissibility for the normal mean and Joseph Hodges and Erich Lehmann's earlier admissibility results frame the boundary between the dimension-one-and-two admissibility of the sample mean and Stein's 1956 inadmissibility result for $p\ge 3$. James Berger's Statistical Decision Theory and Bayesian Analysis ^{[Berger §4.4]} (1985) and Christian Robert's The Bayesian Choice ^{[Robert Ch. 2]} (1994; 2nd ed. 2007) are the standard modern treatments, the latter organising Bayesian estimation explicitly around the loss-then-posterior structure, while Lehmann and Casella's Theory of Point Estimation ^{[Lehmann & Casella §4]} (1998) integrates Bayes estimation with the classical optimality theory and the generalized-Bayes recovery of frequentist rules.

Bibliography Master

@book{wald1950statistical,
  author    = {Wald, Abraham},
  title     = {Statistical Decision Functions},
  publisher = {Wiley},
  year      = {1950}
}

@book{blackwellgirshick1954theory,
  author    = {Blackwell, David and Girshick, Meyer A.},
  title     = {Theory of Games and Statistical Decisions},
  publisher = {Wiley},
  year      = {1954}
}

@book{raiffaschlaifer1961applied,
  author    = {Raiffa, Howard and Schlaifer, Robert},
  title     = {Applied Statistical Decision Theory},
  publisher = {Harvard Business School},
  year      = {1961}
}

@book{berger1985statistical,
  author    = {Berger, James O.},
  title     = {Statistical Decision Theory and Bayesian Analysis},
  edition   = {2nd},
  series    = {Springer Series in Statistics},
  publisher = {Springer},
  year      = {1985}
}

@book{robert2007bayesian,
  author    = {Robert, Christian P.},
  title     = {The Bayesian Choice},
  edition   = {2nd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {2007}
}

@book{lehmanncasella1998theory,
  author    = {Lehmann, Erich L. and Casella, George},
  title     = {Theory of Point Estimation},
  edition   = {2nd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {1998}
}

@book{casellaberger2002statistical,
  author    = {Casella, George and Berger, Roger L.},
  title     = {Statistical Inference},
  edition   = {2nd},
  publisher = {Duxbury},
  year      = {2002}
}