The Neyman-Pearson Lemma and Most-Powerful Tests

Intuition Beginner

A hypothesis test is a rule for deciding between two stories about where your data came from. Maybe a coin is fair, or maybe it is biased. Maybe a new drug does nothing, or maybe it helps. You collect data, and you need a rule that turns the data into one of two verdicts: reject the first story, or do not reject it. The question this unit answers is: among all such rules, which one is the best, and what does "best" even mean?

Any rule can make two kinds of mistake. You can reject the first story when it was true (a false alarm), or fail to reject it when the second story was real (a missed detection). With limited data you cannot drive both to zero at once. So the standard move is to cap the false-alarm rate at a small number you choose, like 5 in 100, and then make the missed-detection rate as small as possible.

The chance of correctly catching the second story when it is true is called the power of the test. So the goal becomes: among all rules whose false-alarm rate is at most your cap, pick the one with the highest power. Such a rule is called most powerful.

The remarkable answer, found by Neyman and Pearson, is that for the simplest case — one specific first story against one specific second story — the best rule has a single clean form. You compare how likely your data are under the second story versus the first, forming a ratio, and you reject when that ratio is large enough. Nothing more clever beats it.

Visual Beginner

Figure: two bell-shaped density curves on one data axis. The left curve is the data distribution if the first story (the null) is true; the right curve is the distribution if the second story (the alternative) is true. A vertical cutoff sits between them, and data to its right make you reject the first story. The left-curve area past the cutoff is the false-alarm rate (the size); the right-curve area past the cutoff is the power. Sliding the cutoff left raises both areas; sliding it right lowers both. The Neyman-Pearson rule places the cutoff where the false-alarm area equals your chosen cap, and there the power is as large as it can be.

 density
  |        null story            alternative story
  |        (story 1)              (story 2)
  |          .-.                       .-.
  |         /   \                     /   \
  |        /     \      cutoff       /     \
  |       /       \       |         /       \
  |      /         \      |        /         \
  |     /           \.    |      ./           \
  |    /              \.  |    ./               \
  +---------------------------------------------------> data
                          |<--- reject story 1 (data here) --->
        left-of-curve area right of cutoff = false-alarm rate (size)
        right-of-curve area right of cutoff = power

Worked example Beginner

A factory line produces rods whose length scatters around a target with a known wobble. Under the current setting the average length is $\theta =10$; a worry is that the line has drifted to $\theta =11$. You measure one rod, getting a value $X$, and the wobble has standard deviation $2$. You want to test "the average is $10$" against "the average is $11$", with a false-alarm cap of about $5$ in $100$.

Step 1. Which way should the rule point? The drifted setting pushes lengths higher, so a large $X$ is evidence for $\theta =11$. The rule will reject "average is $10$" when $X$ is bigger than some cutoff $c$.

Step 2. Set the cutoff from the false-alarm cap. If the true average is $10$, then $X$ is centered at $10$ with spread $2$. We want the chance that $X$ exceeds $c$ to be $5$ in $100$. The value that leaves $5$ in $100$ in the upper tail of a bell curve sits about $1.645$ spreads above center, so $c=10+1.645\times 2=13.29$.

Step 3. Compute the power. If the true average is really $11$, then $X$ is centered at $11$ with spread $2$. The power is the chance $X$ exceeds $13.29$. That is $(13.29-11)/2=1.145$ spreads above the alternative center, and the upper-tail area past $1.145$ is about $0.126$.

What this tells us. With a single rod the test rejects only about $13$ times in $100$ when the line truly drifted — a weak test, because one measurement barely separates a center of $10$ from a center of $11$. The fix is more data: averaging many rods shrinks the spread, pushes the two bell curves apart, and drives the power up. The shape of the rule, though, never changes: reject when the measurement is large enough.

Check your understanding Beginner

Formal definition Intermediate+

Fix a sample space $(\mathcal{X},\mathcal{B})$ and a family of probability measures $\{P_{\theta }:\theta \in \Theta \}$ on it (the measure-theoretic setting of 37.01.01). A hypothesis-testing problem partitions $\Theta ={\Theta }_0\cup {\Theta }_1$ into a null ${\Theta }_0$ and an alternative ${\Theta }_1$. A hypothesis is simple if its parameter set is a singleton and composite otherwise.

Definition (test function). A (possibly randomized) test is a measurable function $\phi :\mathcal{X}\to [0,1]$; on observing $x$ one rejects the null with probability $\phi (x)$ (drawing an independent coin). A non-randomized test takes values in $\{0,1\}$ and corresponds to a rejection region $R=\{x:\phi (x)=1\}$. Randomization is the price of achieving an exact error level when the data law has atoms.

Definition (power function, size, level). The power function of $\phi$ is

$${\beta }_{\phi }(\theta )={\mathbb{E}}_{\theta }[\phi (X)]={\int }_{\mathcal{X}}\phi (x)d{P}_{\theta }(x),$$

the probability of rejection at $\theta$. The size of $\phi$ is ${\sup }_{\theta \in {\Theta }_0}{\beta }_{\phi }(\theta )$. The test has level $\alpha$ if its size is at most $\alpha$. The type I error at $\theta \in {\Theta }_0$ is ${\beta }_{\phi }(\theta )$ (rejecting a true null); the type II error at $\theta \in {\Theta }_1$ is $1-{\beta }_{\phi }(\theta )$ (failing to reject a false null). For an alternative $\theta \in {\Theta }_1$, ${\beta }_{\phi }(\theta )$ is the power at $\theta$.

Definition (most powerful test). Within the class of level-$\alpha$ tests for $H_0:\theta \in {\Theta }_0$ against $H_1:\theta \in {\Theta }_1$, a test ${\phi }^{\ast }$ is most powerful (MP) at ${\theta }_1\in {\Theta }_1$ if ${\beta }_{{\phi }^{\ast }}({\theta }_1)\ge {\beta }_{\phi }({\theta }_1)$ for every level-$\alpha$ test $\phi$. When ${\Theta }_0=\{{\theta }_0\}$ and ${\Theta }_1=\{{\theta }_1\}$ are both simple, an MP test is just one maximizing power among all $\phi$ with ${\beta }_{\phi }({\theta }_0)\le \alpha$ ^{[Casella & Berger §8.3.2]}.

Definition (likelihood ratio and the Neyman-Pearson test). Let $p_0,p_1$ be densities of $P_{{\theta }_0},P_{{\theta }_1}$ with respect to a common dominating measure $\mu$ (e.g. $\mu =P_{{\theta }_0}+P_{{\theta }_1}$, by Radon-Nikodym; the likelihood object of 45.01.02). The likelihood ratio is $\Lambda (x)=p_1(x)/p_0(x)$ (set $\Lambda =+\infty$ where $p_0=0<p_1$). A Neyman-Pearson (NP) test with critical value $k\in [0,\infty )$ and randomization $\gamma \in [0,1]$ is

$$\boxed{{\phi }_{k,\gamma }(x)=\{\begin{array}{ll}1& \text{if }{p}_1(x)>k{p}_0(x),\\ \gamma & \text{if }{p}_1(x)=k{p}_0(x),\\ 0& \text{if }{p}_1(x)<k{p}_0(x).\end{array}}$$

The test rejects when the data are sufficiently more likely under $H_1$ than under $H_0$, breaking ties on the boundary $\{p_1=k{p}_0\}$ with a coin of probability $\gamma$.

Counterexamples to common slips

• Size is a supremum over the null, not the value at one point. For a composite null, capping ${\beta }_{\phi }({\theta }_0)$ at one ${\theta }_0$ is not enough; the size is ${\sup }_{\theta \in {\Theta }_0}{\beta }_{\phi }(\theta )$, and a test can have small power at one null point yet exceed $\alpha$ elsewhere on ${\Theta }_0$.
• Randomization is not optional for exact size with atoms. For discrete data (binomial, Poisson) the attainable non-randomized sizes are a finite set; the cutoff $k$ alone cannot hit an arbitrary $\alpha$, and the boundary probability $\gamma$ is what makes ${\beta }_{\phi }({\theta }_0)=\alpha$ exactly. Without it the NP lemma's "most powerful at exactly level $\alpha$" claim fails.
• The likelihood ratio thresholds $p_1/p_0$, not $p_0/p_1$. Rejecting when the ratio is small gives the worst test, not the best. The reject region is where the alternative outvotes the null.

Key theorem with proof Intermediate+

The fundamental lemma has three parts: an NP test is most powerful at its level (sufficiency), such a test exists at every level (existence with randomization), and every most-powerful test is an NP test off the boundary (necessity).

Theorem (Neyman-Pearson fundamental lemma). Let $H_0:P_{{\theta }_0}$ versus $H_1:P_{{\theta }_1}$ be simple, with densities $p_0,p_1$ relative to $\mu$, and fix $\alpha \in [0,1]$.

(i) (Existence.) There exist $k\in [0,\infty )$ and $\gamma \in [0,1]$ such that the NP test ${\phi }_{k,\gamma }$ has size exactly $\alpha$: ${\beta }_{{\phi }_{k,\gamma }}({\theta }_0)={\mathbb{E}}_{{\theta }_0}[{\phi }_{k,\gamma }]=\alpha$.

(ii) (Sufficiency.) Any NP test ${\phi }_{k,\gamma }$ with ${\mathbb{E}}_{{\theta }_0}[{\phi }_{k,\gamma }]=\alpha$ is most powerful at level $\alpha$ for $H_0$ against $H_1$.

(iii) (Necessity.) If ${\phi }^{\ast }$ is most powerful at level $\alpha$, then ${\phi }^{\ast }$ is an NP test: there is a $k$ with ${\phi }^{\ast }(x)=1$ when $p_1(x)>k{p}_0(x)$ and ${\phi }^{\ast }(x)=0$ when $p_1(x)<k{p}_0(x)$, for $\mu$-almost every $x$ (it may differ from ${\phi }_{k,\gamma }$ only on the boundary $\{p_1=k{p}_0\}$). Unless there is a level-$\alpha$ test of power $1$, ${\phi }^{\ast }$ also has size exactly $\alpha$.

Proof of (i). Define $F(c)=P_{{\theta }_0}(\Lambda (X)>c)$ for $c\in [0,\infty )$, where $\Lambda =p_1/p_0$. As a survival function of $\Lambda$ under $P_{{\theta }_0}$, $F$ is nonincreasing and right-continuous, with $F(0^{-})=1$ and $F(\infty )=0$. If $\alpha =0$ take $k=\infty$ effectively (reject only where $p_0=0<p_1$); assume $\alpha \in (0,1]$. Choose

$$k=\inf \{c\ge 0:F(c)\le \alpha \}.$$

By right-continuity $P_{{\theta }_0}(\Lambda >k)\le \alpha \le P_{{\theta }_0}(\Lambda \ge k)$. Put the leftover mass on the boundary: set

$$\gamma =\frac{\alpha -P_{{\theta }_0}(\Lambda >k)}{P_{{\theta }_0}(\Lambda =k)}$$

when $P_{{\theta }_0}(\Lambda =k)>0$, and $\gamma =0$ otherwise (then $P_{{\theta }_0}(\Lambda >k)=\alpha$ already). Then $\gamma \in [0,1]$ and

$${\mathbb{E}}_{{\theta }_0}[{\phi }_{k,\gamma }]=P_{{\theta }_0}(\Lambda >k)+\gamma P_{{\theta }_0}(\Lambda =k)=\alpha ,$$

so the NP test has size exactly $\alpha$. $\square$

Proof of (ii). Let $\phi ={\phi }_{k,\gamma }$ have ${\mathbb{E}}_{{\theta }_0}\phi =\alpha$, and let $\psi$ be any test with ${\mathbb{E}}_{{\theta }_0}\psi \le \alpha$. Consider the pointwise product

$$(\phi (x)-\psi (x))(p_1(x)-k{p}_0(x))\ge 0\text{for all }x.$$

Indeed, where $p_1>k{p}_0$ the test $\phi =1\ge \psi$, so the first factor is $\ge 0$ and the second is $>0$; where $p_1<k{p}_0$ the test $\phi =0\le \psi$, so the first factor is $\le 0$ and the second is $<0$; where $p_1=k{p}_0$ the second factor vanishes. Integrating the inequality against $\mu$,

$$0\le \int (\phi -\psi )(p_1-k{p}_0)d\mu =({\beta }_{\phi }({\theta }_1)-{\beta }_{\psi }({\theta }_1))-k({\beta }_{\phi }({\theta }_0)-{\beta }_{\psi }({\theta }_0)).$$

Rearranging, ${\beta }_{\phi }({\theta }_1)-{\beta }_{\psi }({\theta }_1)\ge k({\beta }_{\phi }({\theta }_0)-{\beta }_{\psi }({\theta }_0))=k(\alpha -{\mathbb{E}}_{{\theta }_0}\psi )\ge 0$, since $k\ge 0$ and ${\mathbb{E}}_{{\theta }_0}\psi \le \alpha$. Hence ${\beta }_{\phi }({\theta }_1)\ge {\beta }_{\psi }({\theta }_1)$: $\phi$ is most powerful. $\square$

Proof of (iii). Let ${\phi }^{\ast }$ be MP at level $\alpha$, and let $\phi ={\phi }_{k,\gamma }$ be the size-$\alpha$ NP test from (i). By (ii) $\phi$ is MP, so ${\beta }_{{\phi }^{\ast }}({\theta }_1)={\beta }_{\phi }({\theta }_1)$ (both maximal). Re-examine the integral in (ii) with the roles of $\phi$ and $\psi ={\phi }^{\ast }$: the integrand $(\phi -{\phi }^{\ast })(p_1-k{p}_0)$ is everywhere $\ge 0$, yet its integral is $({\beta }_{\phi }({\theta }_1)-{\beta }_{{\phi }^{\ast }}({\theta }_1))-k(\alpha -{\mathbb{E}}_{{\theta }_0}{\phi }^{\ast })\le 0-0=0$ (the first bracket vanishes; the second is $\ge 0$ since ${\mathbb{E}}_{{\theta }_0}{\phi }^{\ast }\le \alpha$, multiplied by $k\ge 0$ and subtracted). A nonnegative integrand with integral $\le 0$ is zero $\mu$-almost everywhere, so $(\phi -{\phi }^{\ast })(p_1-k{p}_0)=0$ a.e. On the set $\{p_1\ne k{p}_0\}$ this forces ${\phi }^{\ast }=\phi$, i.e. ${\phi }^{\ast }=1$ where $p_1>k{p}_0$ and ${\phi }^{\ast }=0$ where $p_1<k{p}_0$; thus ${\phi }^{\ast }$ is an NP test, possibly differing from $\phi$ only on the boundary. Finally, if ${\mathbb{E}}_{{\theta }_0}{\phi }^{\ast }<\alpha$ then $k(\alpha -{\mathbb{E}}_{{\theta }_0}{\phi }^{\ast })>0$ unless $k=0$; but $k=0$ makes ${\phi }^{\ast }$ reject wherever $p_1>0$, giving power $1$. So outside that degenerate case ${\phi }^{\ast }$ has size exactly $\alpha$. $\square$

Bridge. This lemma builds toward the one-sided composite theory of 45.02.02, where a single likelihood-ratio threshold is uniformly optimal across a whole family, and it appears again in the likelihood-ratio and uniformly-most-powerful constructions throughout testing. This is exactly the decision-theoretic Bayes rule of 45.01.01 for the two-action problem under weighted 0-1 loss: thresholding $p_1/p_0$ at $k$ is choosing the action of smaller posterior expected loss, with $k$ encoding the loss ratio and the prior odds, so the most powerful test and the Bayes test are one object viewed through two principles. The central insight is that optimizing a linear functional (power) subject to one linear constraint (size) has an extremal solution of bang-bang form — reject fully, abstain fully, randomize only on the boundary — which is the foundational reason the optimal test is a thresholded ratio rather than anything smoother. Putting these together, sufficiency and necessity pin the MP test to the likelihood-ratio family up to boundary behaviour, and existence supplies the boundary randomization that hits an arbitrary $\alpha$ even with atoms; the bridge is the variational inequality $(\phi -\psi )(p_1-k{p}_0)\ge 0$, which generalises in the master section to the multi-constraint generalized lemma.

Exercises Intermediate+

Advanced results Master

The generalized Neyman-Pearson lemma

The single-constraint lemma extends to several side constraints. Let $f_1,\dots ,f_{m+1}$ be $\mu$-integrable functions on $\mathcal{X}$, and seek a test $\phi :\mathcal{X}\to [0,1]$ maximizing $\int \phi f_{m+1}d\mu$ subject to $\int \phi f_{i}d\mu =c_i$ for $i=1,\dots ,m$. If there exist constants $k_1,\dots ,k_m$ and a test ${\phi }^{\ast }$ satisfying the constraints with

$${\phi }^{\ast }(x)=\{\begin{array}{ll}1& f_{m+1}(x)>{\sum }_{i=1}^m{k}_i{f}_i(x),\\ 0& f_{m+1}(x)<{\sum }_{i=1}^m{k}_i{f}_i(x),\end{array}$$

then ${\phi }^{\ast }$ maximizes $\int \phi f_{m+1}d\mu$ among all tests meeting the constraints; if moreover all $k_i\ge 0$, the constraints may be relaxed to $\int \phi f_i\le c_i$ ^{[Lehmann & Romano §3.5]}. The proof is the same variational identity: $\int ({\phi }^{\ast }-\phi )(f_{m+1}-{\sum }_i{k}_i{f}_i)d\mu \ge 0$ for any competing $\phi$, and the constraints make the $f_i$-terms cancel. Setting $m=1$, $f_1=p_0$, $f_2=p_1$, $c_1=\alpha$ recovers the fundamental lemma; the multiplier $k_1=k$ is the Lagrange multiplier of the size constraint. This is the testing instance of Lagrangian duality: the most powerful test is the unconstrained maximizer of power minus $k$ times size, with $k$ tuned so the size constraint binds at $\alpha$.

Monotone likelihood ratio and the route to composite alternatives

A family $\{p_{\theta }\}$ has monotone likelihood ratio (MLR) in a statistic $T$ if for every ${\theta }^{\prime }>\theta$ the ratio $p_{{\theta }^{\prime }}(x)/p_{\theta }(x)$ is a nondecreasing function of $T(x)$. One-parameter exponential families $p_{\theta }(x)=h(x)\exp (\eta (\theta )T(x)-A(\theta ))$ with $\eta$ increasing have MLR in $T$ ^{[Casella & Berger §8.3.2]}. When MLR holds, the NP test built for a single pair $({\theta }_0,{\theta }_1)$ with ${\theta }_1>{\theta }_0$ has a rejection region $\{T>c\}$ that does not depend on the particular ${\theta }_1$ — only on its being above ${\theta }_0$. This is the mechanism by which a most-powerful-for-one-alternative test becomes uniformly most powerful for the whole one-sided family $H_1:\theta >{\theta }_0$, the Karlin-Rubin theorem developed in 45.02.02. The fundamental lemma thus does double duty: it is both the simple-vs-simple optimum and, through MLR, the engine of the composite one-sided theory.

The p-value, its null distribution, and the duality with confidence sets

For a test statistic $T$ with continuous null distribution, the p-value $p(X)=P_{{\theta }_0}(T(X)\ge T(x))$ (large $T$ extreme) is $\mathrm{Unif}[0,1]$ under $H_0$, so "reject when $p\le \alpha$" has size exactly $\alpha$ and the p-value reports the smallest level at which the data reject. With atoms the p-value is stochastically larger than uniform — $P_{{\theta }_0}(p(X)\le u)\le u$ — so the naive p-value test is conservative, and exact size requires randomization exactly as in the discrete NP construction. Inverting a level-$\alpha$ test family gives a $1-\alpha$ confidence set $\{{\theta }_0:H_0({\theta }_0)\text{ not rejected}\}$; the most-powerful-test optimality transfers to a corresponding optimality of the dual interval, the link that organizes the rest of the chapter. The p-value is not the probability that $H_0$ is true; it is a tail probability computed under $H_0$, and its calibration is the uniform-distribution fact.

Tests as a decision problem: the Bayes reading

Casting $H_0$ versus $H_1$ as a two-action problem with action space $\{$accept, reject$\}$ and loss $L({\theta }_0,\text{reject})=w_I$, $L({\theta }_1,\text{accept})=w_{II}$, zero for correct decisions, the Bayes risk under a prior $({\pi }_0,{\pi }_1)$ is ${\pi }_0{w}_I{\beta }_{\phi }({\theta }_0)+{\pi }_1{w}_{II}(1-{\beta }_{\phi }({\theta }_1))$. Minimizing pointwise (the posterior-expected-loss device of 45.01.01) rejects exactly when ${\pi }_1{w}_{II}{p}_1(x)>{\pi }_0{w}_I{p}_0(x)$, i.e. when $p_1/p_0>k$ with $k={\pi }_0{w}_I/({\pi }_1{w}_{II})$. The Bayes test is an NP test, and as $(\pi ,w)$ range over their values the Bayes tests trace the whole most-powerful family — the size-power frontier is the lower boundary of the convex risk set of tests, its supporting hyperplanes the priors-times-losses. Admissibility of NP tests with $k>0$ (Exercise 8) is the testing case of the admissibility-of-unique-Bayes-rules theorem.

Synthesis. The central insight is that an optimal test maximizes a linear functional (power) under linear constraints (size), so its solution is extremal — bang-bang on the likelihood ratio, randomizing only on the boundary — and this single variational principle is exactly what the fundamental lemma, its generalized multi-constraint form, and the Bayes/decision-theoretic reading all express. The foundational reason the likelihood ratio is privileged is that $k$ is the Lagrange multiplier of the size constraint, so thresholding $p_1/p_0$ is dual to the constrained optimization; the same multiplier reappears as the prior-times-loss odds in the Bayes test of 45.01.01, identifying the most powerful test with the Bayes rule. Putting these together, monotone likelihood ratio lifts the simple-vs-simple optimum to the uniformly-most-powerful one-sided test of 45.02.02, because the rejection region $\{T>c\}$ is alternative-free, and the p-value's uniform null distribution generalises the exact-size guarantee from a single threshold to the whole calibrated reporting scale. The size-power frontier is the lower boundary of the convex risk set of randomized tests, so admissibility, the complete-class structure, and the minimax-Bayes duality of decision theory specialize here verbatim; the bridge across the whole chapter is the test-inversion duality that turns this optimality into optimal confidence sets, where these same objects appear again in interval estimation.

Full proof set Master

Proposition 1 (existence and uniqueness of the critical value). Let $\Lambda =p_1/p_0$ and $\alpha \in (0,1)$. There is a unique $k\ge 0$ and a $\gamma \in [0,1]$ (unique when $P_{{\theta }_0}(\Lambda =k)>0$) such that $P_{{\theta }_0}(\Lambda >k)+\gamma P_{{\theta }_0}(\Lambda =k)=\alpha$.

Proof. The survival function $F(c)=P_{{\theta }_0}(\Lambda >c)$ is nonincreasing and right-continuous with $F(0^{-})\le 1$ and $F(c)\to 0$ as $c\to \infty$. Set $k=\inf \{c\ge 0:F(c)\le \alpha \}$, finite because $F\to 0$. Right-continuity gives $F(k)\le \alpha$, while $F(k^{-})=P_{{\theta }_0}(\Lambda \ge k)\ge \alpha$ by definition of the infimum. Thus $P_{{\theta }_0}(\Lambda >k)\le \alpha \le P_{{\theta }_0}(\Lambda \ge k)$, and the gap $P_{{\theta }_0}(\Lambda \ge k)-P_{{\theta }_0}(\Lambda >k)=P_{{\theta }_0}(\Lambda =k)$. If this gap is $0$, then $P_{{\theta }_0}(\Lambda >k)=\alpha$ and $\gamma$ is irrelevant; otherwise $\gamma =(\alpha -P_{{\theta }_0}(\Lambda >k))/P_{{\theta }_0}(\Lambda =k)\in [0,1]$ is the unique solution. Uniqueness of $k$: any $k^{\prime }<k$ has $F(k^{\prime })>\alpha$ so cannot give size $\le \alpha$ with $\gamma \le 1$; any $k^{\prime }>k$ has $F(k^{\prime -})\le F(k)\le \alpha$ forcing $\gamma <0$ to reach $\alpha$ unless $F(k^{\prime })=\alpha$ on a flat stretch, which the infimum excludes as the canonical choice. $\square$

Proposition 2 (the fundamental variational identity). For any two tests $\phi ,\psi$ and any $k\ge 0$,

$$\int (\phi -\psi )(p_1-k{p}_0)d\mu =({\beta }_{\phi }({\theta }_1)-{\beta }_{\psi }({\theta }_1))-k({\beta }_{\phi }({\theta }_0)-{\beta }_{\psi }({\theta }_0)),$$

and if $\phi ={\phi }_{k,\gamma }$ is an NP test the integrand is $\ge 0$ pointwise.

Proof. Linearity of the integral and ${\beta }_{\phi }({\theta }_i)=\int \phi p_{i}d\mu$ give the displayed equation directly. For the sign: where $p_1>k{p}_0$ the NP test has $\phi =1$, the maximum, so $\phi -\psi \ge 0$ while $p_1-k{p}_0>0$; where $p_1<k{p}_0$, $\phi =0$, so $\phi -\psi \le 0$ while $p_1-k{p}_0<0$; where $p_1=k{p}_0$ the second factor is $0$. In every case the product is $\ge 0$. $\square$

Proposition 3 (sufficiency: NP is MP). If $\phi ={\phi }_{k,\gamma }$ has size $\alpha$ and $\psi$ has size $\le \alpha$, then ${\beta }_{\phi }({\theta }_1)\ge {\beta }_{\psi }({\theta }_1)$.

Proof. By Proposition 2 the integral $\int (\phi -\psi )(p_1-k{p}_0)d\mu \ge 0$, so ${\beta }_{\phi }({\theta }_1)-{\beta }_{\psi }({\theta }_1)\ge k({\beta }_{\phi }({\theta }_0)-{\beta }_{\psi }({\theta }_0))=k(\alpha -{\beta }_{\psi }({\theta }_0))\ge 0$, using $k\ge 0$ and ${\beta }_{\psi }({\theta }_0)\le \alpha$. $\square$

Proposition 4 (necessity: MP is NP a.e.). If $\psi$ is MP at level $\alpha$ and $\phi ={\phi }_{k,\gamma }$ is the size-$\alpha$ NP test, then $\psi =\phi$ on $\{p_1\ne k{p}_0\}$ up to a $\mu$-null set.

Proof. Both maximize power, so ${\beta }_{\psi }({\theta }_1)={\beta }_{\phi }({\theta }_1)$. By Proposition 2 applied with the pair $(\phi ,\psi )$, the integral equals $({\beta }_{\phi }({\theta }_1)-{\beta }_{\psi }({\theta }_1))-k(\alpha -{\beta }_{\psi }({\theta }_0))=0-k(\alpha -{\beta }_{\psi }({\theta }_0))\le 0$ since ${\beta }_{\psi }({\theta }_0)\le \alpha$ and $k\ge 0$. But the integrand is $\ge 0$ pointwise (Proposition 2), so its integral is $\ge 0$; hence the integral is $0$ and the nonnegative integrand vanishes $\mu$-a.e. On $\{p_1\ne k{p}_0\}$ the factor $(p_1-k{p}_0)\ne 0$, forcing $\phi -\psi =0$ there a.e. $\square$

Proposition 5 (uniform distribution of the p-value under a continuous null). Let $T(X)$ have continuous distribution under $P_{{\theta }_0}$, with survival function $G(t)=P_{{\theta }_0}(T\ge t)$, and define $p(X)=G(T(X))$. Then $p(X)\sim \mathrm{Unif}[0,1]$ under $P_{{\theta }_0}$.

Proof. Continuity of the law of $T$ makes $G$ continuous; on the support $G$ is nonincreasing with range $[0,1]$. For $u\in (0,1)$ let $t_u=\inf \{t:G(t)\le u\}$; continuity gives $G(t_u)=u$. Then $\{p(X)\le u\}=\{G(T)\le u\}=\{T\ge t_u\}$ (up to the null event $\{T=t_u\}$, which has probability $0$), so $P_{{\theta }_0}(p(X)\le u)=P_{{\theta }_0}(T\ge t_u)=G(t_u)=u$. Thus the cdf of $p(X)$ is the identity on $[0,1]$. $\square$

Connections Master

• The one-sided composite theory of uniformly most powerful tests 45.02.02 is the direct continuation: when the family has monotone likelihood ratio in a sufficient statistic, the single-pair Neyman-Pearson rejection region $\{T>c\}$ is alternative-free and therefore uniformly most powerful for $H_1:\theta >{\theta }_0$, so the lemma proved here is the generating case of the Karlin-Rubin theorem.

• Statistical decision theory 45.01.01 supplies the frame in which the most powerful test is the Bayes rule for the two-action problem under weighted 0-1 loss; the critical value $k$ is the prior-times-loss odds ratio, the size-power frontier is the lower boundary of the convex risk set of tests, and admissibility of NP tests is the testing instance of the admissibility of unique Bayes rules.

• Sufficiency and the factorization theorem 45.01.02 guarantee that the likelihood ratio, hence the most powerful test, depends on the data only through a sufficient statistic; the exponential-family structure that makes the worked normal and exponential examples tractable is the same structure that produces monotone likelihood ratio downstream.

• The measure-theoretic foundations 37.01.01 underwrite every object here: a test is a measurable $[0,1]$-valued function, size and power are integrals of it against the parameter-indexed measures, the likelihood ratio is a Radon-Nikodym derivative with respect to a common dominating measure, and the boundary-randomization construction is an explicit handling of the atoms those measures may carry.

Historical & philosophical context Master

The framework is due to Jerzy Neyman and Egon Pearson, whose 1933 paper On the problem of the most efficient tests of statistical hypotheses ^{[Neyman & Pearson 1933]} (Phil. Trans. R. Soc. Lond. A 231, 289-337) introduced the two-error formulation, the power function, and the fundamental lemma, deliberately breaking with R. A. Fisher's significance-testing program by insisting that a test be judged against a specified alternative and by its operating characteristics rather than by a single p-value. The Neyman-Pearson program reframed testing as a behavioral rule with controlled long-run error rates, a stance Neyman defended as "inductive behavior" against Fisher's "inductive inference"; the resulting dispute over the meaning of a test ran for decades and still structures the frequentist-Bayesian and Fisher-Neyman debates in the foundations of statistics.

Abraham Wald's decision-theoretic synthesis ^{[Lehmann & Romano §3.1]} (1950) absorbed the lemma as the Bayes rule of a two-action problem, and Erich Lehmann's Testing Statistical Hypotheses (1959; 3rd edition with Joseph Romano, 2005) became the canonical development, extending the lemma through the generalized multi-constraint form, monotone likelihood ratio, unbiased and invariant tests, and the test-confidence-set duality. The randomization device, an artifact of insisting on exact size for discrete data, was contributed by the Neyman-Pearson formulation itself and remains the cleanest illustration of why test functions, not rejection regions, are the right primitive object.

Bibliography Master

@article{neymanpearson1933problem,
  author    = {Neyman, Jerzy and Pearson, Egon S.},
  title     = {On the problem of the most efficient tests of statistical hypotheses},
  journal   = {Philosophical Transactions of the Royal Society of London. Series A},
  volume    = {231},
  pages     = {289--337},
  year      = {1933}
}

@book{lehmannromano2005testing,
  author    = {Lehmann, Erich L. and Romano, Joseph P.},
  title     = {Testing Statistical Hypotheses},
  edition   = {3rd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {2005}
}

@book{casellaberger2002statistical,
  author    = {Casella, George and Berger, Roger L.},
  title     = {Statistical Inference},
  edition   = {2nd},
  publisher = {Duxbury},
  year      = {2002}
}

@book{lehmanncasella1998theory,
  author    = {Lehmann, Erich L. and Casella, George},
  title     = {Theory of Point Estimation},
  edition   = {2nd},
  series    = {Springer Texts in Statistics},
  publisher = {Springer},
  year      = {1998}
}

@book{wald1950statistical,
  author    = {Wald, Abraham},
  title     = {Statistical Decision Functions},
  publisher = {Wiley},
  year      = {1950}
}