45.05.07 · mathematical-statistics / 05-empirical-processes-nonparametric

Linear Rank Statistics and Locally Most Powerful Rank Tests

shipped3 tiersLean: none

Anchor (Master): Hájek & Šidák 1967 Theory of Rank Tests (Academic Press) Ch. II (rank distribution theory), Ch. V (the projection / Hájek-projection lemma and asymptotic normality under alternatives), Ch. VI (contiguity), Ch. VII (locally most powerful rank tests and asymptotically optimal scores); Hájek, Šidák & Sen 1999 Theory of Rank Tests 2e (Academic Press); van der Vaart 1998 Asymptotic Statistics (Cambridge) Ch. 13-14 (rank tests and their asymptotic efficiency); Hájek 1968 Asymptotic normality of simple linear rank statistics under alternatives (Ann. Math. Statist. 39)

Intuition Beginner

Suppose you measure something and you do not trust the scale. The numbers might be off by some unknown stretch or warp, but the order of the numbers — who came first, who came last — you trust completely. Rank methods throw away the raw measurements and keep only this order. Each observation is replaced by its position in the sorted list: the smallest gets rank one, the next gets rank two, and so on. A great deal of honest inference can be done with the ranks alone.

Why is that a good idea? Because the ranks of a shuffled deck do not know what the deck is made of. If the data are just noise with no real signal, then every ordering of the observations is equally likely, no matter what shape the noise has. That single fact — every order equally likely under "nothing is going on" — lets you compute exact error rates for a test without ever knowing the distribution behind the data. A test built this way is called distribution-free: its false-alarm rate is the same whether the noise is bell-shaped, heavy-tailed, or lopsided.

A linear rank statistic is the simplest useful thing you can build from ranks. You attach a weight to each observation (which group it is in, or where it sits along some predictor) and a score to each rank position, then add up weight times score across the sample. Different choices of score give different classic tests: Wilcoxon's test, the normal-scores test, the sign test. The art is choosing the score that is most sensitive to the kind of departure you care about.

The takeaway: replace data by ranks, exploit that all orderings are equally likely under the null, and combine ranks linearly with well-chosen weights and scores to get an exact, distribution-free, and tunable test.

Visual Beginner

Picture eight measurements from two groups, group A and group B, four each. Sort all eight together and read off who lands where. The rank-sum statistic for group B is the total of the rank positions occupied by the B observations. If B tends to be larger, B grabs the high positions and the rank sum is big; if the groups are alike, B lands in the middle and the rank sum is moderate.

sorted position (rank)	1	2	3	4	5	6	7	8
which group landed here	A	A	B	A	B	B	A	B
score (Wilcoxon = the rank)	1	2	3	4	5	6	7	8

Here group B occupies positions $3, 5, 6, 8$ , so the rank sum for B is $3 + 5 + 6 + 8 = 22$ . The largest it could be is $5 + 6 + 7 + 8 = 26$ (B takes the top four), the smallest is $1 + 2 + 3 + 4 = 10$ . A value of $22$ leans toward "B is larger," and you can count how many of the equally likely arrangements give a rank sum that extreme to get an exact p-value.

The picture is the whole idea: forget the measured values, keep the sorted positions, weight each observation by its group, score each position by its rank, and add. The result is a linear rank statistic.

Worked example Beginner

We compute a Wilcoxon two-sample rank statistic by hand and find its exact null mean. Two small groups: group A has values $2, 5$ and group B has values $7, 11$ . So group sizes are $m = 2$ and $n = 2$ , four observations in all.

Step 1. Pool and sort. The pooled, sorted values are $2, 5, 7, 11$ , getting ranks $1, 2, 3, 4$ .

Step 2. Read off the ranks of group B. Group B's values $7$ and $11$ sit at ranks $3$ and $4$ .

Step 3. Form the rank sum for B. The Wilcoxon statistic is the sum of B's ranks: $W = 3 + 4 = 7$ .

Step 4. Find the null mean. Under the null that all four observations come from the same source, every choice of which two ranks belong to B is equally likely. There are six such choices. Their rank sums are: ranks ${1, 2} \to 3$ ; ${1, 3} \to 4$ ; ${1, 4} \to 5$ ; ${2, 3} \to 5$ ; ${2, 4} \to 6$ ; ${3, 4} \to 7$ . The average of $3, 4, 5, 5, 6, 7$ is $30/6 = 5$ . So the null mean of $W$ is $5$ .

Step 5. Compare. The observed $W = 7$ is the largest possible value, achieved by exactly one of the six arrangements. So the one-sided exact p-value for "B is larger" is $1/6 \approx 0.167$ .

What this tells us: we computed both the statistic and its exact reference distribution using only counting — no formula for the underlying spread or shape was ever needed. The null mean $5$ matches the simple formula "average rank times group size," namely $\frac{4 + 1}{2} \times 2 = 5$ , and the whole reference distribution is the list of six equally likely rank sums.

Check your understanding Beginner

Exercise (easy, multiple choice).

What does it mean for a rank test to be distribution-free under the null?

A. The test never makes an error B. The null distribution of the test statistic does not depend on the unknown distribution generating the data C. The data must be uniformly distributed D. The ranks are spread out evenly across all values

Hint

Under "nothing is going on," every ordering of the observations is equally likely whatever the underlying shape; what does that buy you for the statistic's reference distribution?

Answer

B. Distribution-free means the null sampling distribution of the statistic is the same regardless of the underlying continuous distribution, because under the null all orderings of the data are equally likely. Feedback-correct: this is exactly what lets you tabulate exact p-values once and use them for any continuous data. Feedback-wrong: A overstates it (tests still err); C and D confuse a property of the ranks with a requirement on the data.

Formal definition Intermediate+

Let $X_{1}, \dots, X_{N}$ be observations and let $R_{i}$ denote the rank of $X_{i}$ in the pooled sample, so $R_{i} = \sum_{j = 1}^{N} 1 {X_{j} \leq X_{i}}$ when the values are distinct. The vector $R = (R_{1}, \dots, R_{N})$ is a random permutation of $(1, \dots, N)$ . The convergence modes $d, P$ and the $o_{P} / O_{P}$ calculus are as in 45.04.07; the Hájek-projection method is the rank-statistic instance of the U-statistic projection of 45.05.05; contiguity and Le Cam's third lemma are as in 45.04.07.

Definition (distribution-freeness of ranks). Under the hypothesis of randomness — $X_{1}, \dots, X_{N}$ i.i.d. from a continuous distribution, equivalently exchangeable with no ties almost surely — the rank vector $R$ is uniformly distributed on the symmetric group $S_{N}$ :

P_{0} (R = r) = \frac{1}{N !}, r \in S_{N} .

Consequently any statistic that is a function of $R$ alone has a null distribution free of the underlying continuous $F$ .

Definition (simple linear rank statistic). Fix regression constants $c_{1}, \dots, c_{N} \in R$ and a score function assigning a score $a_{N} (1), \dots, a_{N} (N)$ to the rank positions. The simple linear rank statistic is

S = S_{N} = i = 1 \sum N c_{i} a_{N} (R_{i}) = i = 1 \sum N c_{i} a_{N} (R_{i}) .

The constants encode the design (group membership or a covariate); the scores encode the alternative shape one is tuned against. Writing $\overset{c}{ˉ} = N^{- 1} \sum_{i} c_{i}$ and $\overset{a}{ˉ} = N^{- 1} \sum_{j} a_{N} (j)$ , the two-sample (Wilcoxon) case takes $c_{i} = 1 {i \in group 2}$ and $a_{N} (j) = j$ ; the regression case takes $c_{i}$ equal to a covariate value and $a_{N} (\cdot)$ a chosen score.

Definition (scores from a score-generating function). A score function is generated by $φ : (0, 1) \to R$ when $a_{N} (j) = φ (j / (N + 1))$ or, in the smoothed form, $a_{N} (j) = E φ (U_{(j)})$ with $U_{(j)}$ the $j$ -th uniform order statistic. The canonical examples are Wilcoxon scores $φ (u) = u - \frac{1}{2}$ (so $a_{N} (j) \propto j$ ), normal / van der Waerden scores $φ (u) = Φ^{- 1} (u)$ , and sign / median scores $φ (u) = sgn (u - \frac{1}{2})$ .

Counterexamples to common slips Intermediate+

Distribution-freeness is a null property only. Under an alternative the rank vector is no longer uniform on $S_{N}$ , and the statistic's distribution does depend on $F$ ; the exact null calibration is what survives, not the alternative law.
The summands $c_{i} a_{N} (R_{i})$ are not independent. The ranks $R_{i}$ are a permutation, hence strongly dependent (their sum is fixed at $1 + \dots + N$ ). The exact variance is therefore the sampling-without-replacement variance, not $\sum_{i} c_{i}^{2} Var a_{N} (R_{i})$ , and the correction is the content of the variance formula below.
Centring matters. If neither the constants nor the scores are centred, $S$ has a nonzero null mean $N \overset{c}{ˉ} \overset{a}{ˉ}$ ; the genuinely informative object is the centred statistic $\sum_{i} (c_{i} - \overset{c}{ˉ}) (a_{N} (R_{i}) - \overset{a}{ˉ})$ , whose null mean is zero.
The "best" score depends on the alternative shape. Wilcoxon scores are optimal for a logistic shift, normal scores for a Gaussian shift, sign scores for a Laplace shift. Using Wilcoxon scores against a Gaussian alternative is consistent but not locally most powerful; the score must be matched to $f$ .

Key theorem with proof Intermediate+

The exact null moments of a simple linear rank statistic are pure combinatorics — they follow from the uniform law on $S_{N}$ — and the asymptotic normality follows by projecting $S$ onto sums of independent terms, the rank-statistic form of the Hájek projection of 45.05.05 ^{[van der Vaart — Asymptotic Statistics]}.

Theorem (exact null moments and asymptotic normality of a simple linear rank statistic). Let $S = \sum_{i = 1}^{N} c_{i} a_{N} (R_{i})$ with $R$ uniform on $S_{N}$ under the null.

(i) Exact mean. $E_{0} S = N \overset{c}{ˉ} \overset{a}{ˉ} .$

(ii) Exact variance.

Var_{0} (S) = \frac{1}{N - 1} i = 1 \sum N (c_{i} - \overset{c}{ˉ})^{2} j = 1 \sum N (a_{N} (j) - \overset{a}{ˉ})^{2} .

(iii) Asymptotic normality. If the scores are generated by a square-integrable $φ$ that is not constant, $\sum_{i} (c_{i} - \overset{c}{ˉ})^{2} > 0$ , and the Noether condition

\frac{max _{i} ( c _{i} - c ˉ ) ^{2}}{\sum _{i} ( c _{i} - c ˉ ) ^{2}} \to 0

holds, then $(S - E_{0} S) / Var_{0} (S) d N (0, 1)$ .*

Proof. (i). Each $R_{i}$ is marginally uniform on ${1, \dots, N}$ , so $E_{0} a_{N} (R_{i}) = \overset{a}{ˉ}$ for every $i$ , and $E_{0} S = \sum_{i} c_{i} \overset{a}{ˉ} = N \overset{c}{ˉ} \overset{a}{ˉ}$ by linearity.

(ii). Centre both sequences: write $c_{i}^{\circ} = c_{i} - \overset{c}{ˉ}$ and $a_{N}^{\circ} (j) = a_{N} (j) - \overset{a}{ˉ}$ , so $\sum_{i} c_{i}^{\circ} = 0$ and $\sum_{j} a_{N}^{\circ} (j) = 0$ , and $S - E_{0} S = \sum_{i} c_{i}^{\circ} a_{N}^{\circ} (R_{i})$ . Under the uniform law on $S_{N}$ , the variance and covariance of the score variables $a_{N}^{\circ} (R_{i})$ are those of sampling without replacement: $Var_{0} a_{N}^{\circ} (R_{i}) = \frac{1}{N} \sum_{j} a_{N}^{\circ} (j)^{2} =: v$ , and for $i \neq = k$ , $Cov_{0} (a_{N}^{\circ} (R_{i}), a_{N}^{\circ} (R_{k})) = - \frac{1}{N - 1} v$ , since the row sums of the doubly-centred score table force the off-diagonal covariance to be $- v / (N - 1)$ . Therefore

Var_{0} (S) = i \sum (c_{i}^{\circ})^{2} v + i \neq = k \sum c_{i}^{\circ} c_{k}^{\circ} (- \frac{v}{N - 1}) = v [i \sum (c_{i}^{\circ})^{2} - \frac{1}{N - 1} i \neq = k \sum c_{i}^{\circ} c_{k}^{\circ}] .

Because $\sum_{i} c_{i}^{\circ} = 0$ , one has $\sum_{i \neq = k} c_{i}^{\circ} c_{k}^{\circ} = (\sum_{i} c_{i}^{\circ})^{2} - \sum_{i} (c_{i}^{\circ})^{2} = - \sum_{i} (c_{i}^{\circ})^{2}$ , so the bracket equals $\sum_{i} (c_{i}^{\circ})^{2} (1 + \frac{1}{N - 1}) = \frac{N}{N - 1} \sum_{i} (c_{i}^{\circ})^{2}$ . Substituting $v = \frac{1}{N} \sum_{j} a_{N}^{\circ} (j)^{2}$ gives $Var_{0} (S) = \frac{1}{N - 1} \sum_{i} (c_{i}^{\circ})^{2} \sum_{j} a_{N}^{\circ} (j)^{2}$ , the claimed formula.

(iii). Let $F_{N}$ be the pooled empirical distribution function and recall $R_{i} = N F_{N} (X_{i})$ . The projection of $S$ onto sums of independent terms — the Hájek projection of 45.05.05 — replaces $a_{N} (R_{i})$ by its conditional expectation given $X_{i}$ . To leading order $a_{N} (R_{i}) = φ (R_{i} / (N + 1)) \approx φ (F (X_{i}))$ , and the projection is

\hat{S} = N \overset{c}{ˉ} \overset{a}{ˉ} + i = 1 \sum N c_{i}^{\circ} φ (F (X_{i})) + R_{N},

where $φ (F (X_{i}))$ are i.i.d., mean (after centring) zero, variance $\int_{0}^{1} φ^{\circ} (u)^{2} d u$ with $φ^{\circ} = φ - \int φ$ . Hájek's lemma gives $E_{0} (S - \hat{S})^{2} = o (Var_{0} S)$ : the difference between using $φ (R_{i} / (N + 1))$ and $φ (F (X_{i}))$ is a higher-order empirical-process remainder, controlled because $φ$ is square-integrable. The leading term $\sum_{i} c_{i}^{\circ} φ^{\circ} (F (X_{i}))$ is a weighted sum of i.i.d. variables; under the Noether condition no single weight $c_{i}^{\circ}$ dominates, so the Lindeberg condition holds and the Lindeberg-Feller central limit theorem (used in 45.05.05) gives a normal limit. Slutsky's lemma 45.04.07 absorbs the $o_{P} (1)$ remainder, yielding $(S - E_{0} S) / Var_{0} S d N (0, 1)$ . $□$

Bridge. This projection argument builds toward the contiguous-alternative limit and the optimal-score theory of the Master tier, and the very same one-coordinate conditional expectation appears again in the U-statistic asymptotics of 45.05.05, where it linearises a non-linear functional of the sample. The foundational reason the theorem works is that a rank statistic, though built from the dependent permutation $R$ , is in $L^{2}$ dominated by its projection onto sums of the independent scores $φ (F (X_{i}))$ : the dependence among ranks contributes only a vanishing-relative-variance remainder, and the central limit theorem then operates on the surviving i.i.d. average. This is exactly the Hájek-projection method, and it generalises the U-statistic linearisation of 45.05.05 from symmetric kernels to the rank transform. Putting these together, the exact null variance from sampling-without-replacement combinatorics and the asymptotic normality from projection are two readings of one fact — that $S$ centred is a near-linear functional of the empirical distribution — and the bridge to local power is Le Cam's third lemma 45.04.07, which tilts this null Gaussian into its alternative form by a single covariance with the log-likelihood ratio.

Exercises Intermediate+

Exercise 5 (medium, symbolic).

Write down the Hájek projection $\hat{S}$ of $S = \sum_{i} c_{i} a_{N} (R_{i})$ onto sums of single-coordinate functions, and state the $L^{2}$ bound that makes $S$ and $\hat{S}$ share a limit law.

Hint

Project onto $\sum_{i} g (X_{i})$ ; the leading term replaces $a_{N} (R_{i})$ by $φ (F (X_{i}))$ . The projection method of 45.05.05 gives the residual bound.

Answer

The projection is $\hat{S} = E_{0} S + \sum_{i = 1}^{N} (E [S ∣ X_{i}] - E_{0} S)$ , and computing one term using $a_{N} (R_{i}) \approx φ (F (X_{i}))$ gives $\hat{S} = N \overset{c}{ˉ} \overset{a}{ˉ} + \sum_{i} c_{i}^{\circ} φ^{\circ} (F (X_{i})) + o_{P} (\cdot)$ , a sum of independent terms. Hájek's projection lemma supplies $E_{0} (S - \hat{S})^{2} = o (Var_{0} S)$ , equivalently $Var_{0} (S - \hat{S}) / Var_{0} (\hat{S}) \to 0$ . By the projection criterion of 45.05.05, $S$ and $\hat{S}$ standardised have the same limit, so the central limit theorem for the i.i.d. average $\hat{S}$ transfers to $S$ . The conditional expectation onto $σ (X_{i})$ is an $L^{2}$ -orthogonal projection, forcing the Pythagorean variance split.

Exercise 6 (hard, symbolic).

(Local power via the third lemma.) In the two-sample location-shift model with the second group shifted by $h / N$ , the log-likelihood ratio is asymptotically that of a LAN family, and $(S^{*}, Λ_{N}) \to (S^{*}, Λ)$ jointly Gaussian under the null, where $S^{*}$ is the standardised rank statistic. Using Le Cam's third lemma 45.04.07, show that under the contiguous alternative $S^{*} d N (μ, 1)$ with $μ = Cov (S^{*}, Λ)$ , and identify $μ$ for Wilcoxon scores against a logistic shift.

Hint

The third lemma shifts the null mean of $S^{*}$ by its covariance with the log-ratio. For matched score and alternative, that covariance is the standardised noncentrality $h κ$ with $κ$ a Fisher-information-type constant.

Answer

Under the null $(S^\ast, \Lambda_N) \xrightarrow{d} N\big((0, -\tfrac12\sigma^2), \begin{psmallmatrix}1 & c\\ c & \sigma^2\end{psmallmatrix}\big)$ , with $c = Cov (S^{*}, Λ)$ . By Le Cam's third lemma 45.04.07, contiguity of the location-shift alternatives lets us tilt: under $Q_{N}$ , $S^{*} d N (c, 1)$ , so $μ = c$ . The covariance $c$ equals $h$ times the standardised correlation between the score-generating function $φ$ and the alternative's log-derivative $φ_{f} = - f^{'} / f \circ F^{- 1}$ , namely $c = h mn / N \cdot ρ$ with $ρ = \int φ^{\circ} φ_{f}^{\circ} / (\int (φ^{\circ})^{2} \int (φ_{f}^{\circ})^{2})^{1/2}$ in the standardised normalisation. For Wilcoxon scores $φ (u) = u - \frac{1}{2}$ against the logistic shift, $φ_{f} (u) \propto u - \frac{1}{2}$ as well, so $ρ = 1$ and $μ = h mn / N \cdot (\int (φ^{\circ})^{2} \int (φ_{f}^{\circ})^{2})^{1/2}$ is maximal: Wilcoxon is locally most powerful for the logistic. Rubric: full credit for the third-lemma tilt, $μ = Cov (S^{*}, Λ)$ , and the $ρ = 1$ identification.

Exercise 7 (hard, symbolic).

(Optimal scores.) For a one-sample/two-sample location family with density $f$ , the locally most powerful rank test uses scores generated by $φ_{f} (u) = - \frac{f ^{'} ( F ^{- 1} ( u ) )}{f ( F ^{- 1} ( u ) )}$ . Derive $φ_{f}$ for the logistic, the standard normal, and the Laplace densities, and name the resulting tests.

Hint

Compute $- f^{'} / f$ and re-express in terms of $u = F (x)$ . For logistic $- f^{'} / f = 2 F - 1$ ; for normal it is the quantile $Φ^{- 1}$ ; for Laplace it is the sign.

Answer

Logistic. $f (x) = e^{- x} / (1 + e^{- x})^{2}$ , $F (x) = 1/ (1 + e^{- x})$ , and $- f^{'} / f = 1 - 2 F (x)$ up to sign; with $u = F (x)$ , $φ_{f} (u) = 2 u - 1$ , the Wilcoxon scores. Normal. $f = ϕ$ , $- f^{'} / f = x$ , and $x = Φ^{- 1} (u)$ , so $φ_{f} (u) = Φ^{- 1} (u)$ , the van der Waerden / normal-scores test. Laplace. $f (x) = \frac{1}{2} e^{- ∣ x ∣}$ , $- f^{'} / f = sgn (x)$ , and $sgn (x) = sgn (u - \frac{1}{2})$ , so $φ_{f} (u) = sgn (u - \frac{1}{2})$ , the sign / median scores. Each score is the log-derivative of the assumed density pushed through the quantile transform, so the LMP rank test matches its scores to the alternative shape. Rubric: full credit for the three $- f^{'} / f$ computations and the three named tests.

Exercise 8 (hard, symbolic).

(Asymptotic relative efficiency.) The ARE of a rank test with score $φ$ relative to the $t$ -test, against a location alternative with density $f$ of variance $σ_{f}^{2}$ , is $e (φ, t) = σ_{f}^{2} (\int_{0}^{1} φ^{\circ} (u) φ_{f}^{\circ} (u) d u)^{2} / \int_{0}^{1} φ^{\circ} (u)^{2} d u$ . Specialise to Wilcoxon scores ( $φ (u) = u$ ) at a normal alternative and recover the classic value $3/ π \approx 0.955$ , cross-referencing the ARE machinery of 45.04.05.

Hint

For Wilcoxon against normal, the efficiency reduces to $12 σ^{2} (\int f^{2})^{2}$ ; for the standard normal $\int ϕ^{2} = 1/ (2 π)$ .

Answer

The Wilcoxon-to- $t$ ARE against any $f$ with variance $σ^{2}$ equals $12 σ^{2} (\int_{- \infty}^{\infty} f (x)^{2} d x)^{2}$ , obtained by substituting $φ (u) = u$ and $φ_{f} = - f^{'} / f$ into the general formula and integrating by parts (the cross term $\int φ^{\circ} φ_{f}^{\circ}$ becomes $\int f^{2}$ ). For the standard normal, $\int ϕ^{2} = \frac{1}{2 π}$ and $σ^{2} = 1$ , so $e = 12 \cdot 1 \cdot (\frac{1}{2 π})^{2} = 12 \cdot \frac{1}{4 π} = \frac{3}{π} \approx 0.955$ . The Wilcoxon test loses only about $4.5%$ efficiency at the Gaussian while gaining enormously at heavy tails, and the ARE itself is the ratio of the two limiting noncentralities computed by 45.04.05. Rubric: full credit for the $12 σ^{2} (\int f^{2})^{2}$ reduction, the $\int ϕ^{2} = 1/ (2 π)$ value, and the $3/ π$ answer.

Advanced results Master

The exact null moments come from the uniform law on $S_{N}$ ; the asymptotics come from the projection of $S$ onto sums of independent scores; the local power comes from Le Cam's third lemma 45.04.07; and the optimal scores come from differentiating the log-likelihood of the order statistics at the null. These four facts assemble into the complete asymptotic theory of rank tests.

Theorem 1 (Hájek's projection / asymptotic-linearity representation). Let $S = \sum_{i} c_{i} a_{N} (R_{i})$ with scores generated by a square-integrable $φ$ of bounded variation (or absolutely continuous with $\int (φ^{'})^{2} < \infty$ ). Then

S = E_{0} S + i = 1 \sum N c_{i}^{\circ} φ (F (X_{i})) + R_{N}, E_{0} R_{N}^{2} = o (Var_{0} S),

so $S$ is asymptotically linear in the i.i.d. scores $φ (F (X_{i}))$ ; this is the rank-statistic instance of the Hájek projection of 45.05.05. The leading sum drives both the null central limit theorem and, after a contiguous change of measure, the alternative limit ^{[Hájek, J. — Asymptotic normality of simple linear rank statistics under alternatives]}.

Theorem 2 (limit under contiguous alternatives via the third lemma). In the location-shift family with second-group shift $h / N$ , the local alternatives are contiguous to the null (LAN log-ratio $Λ_{N} d N (- \frac{1}{2} σ^{2}, σ^{2})$ , 45.04.07). The standardised statistic $S^{*} = (S - E_{0} S) / Var_{0} S$ and $Λ_{N}$ are jointly asymptotically Gaussian under the null, and Le Cam's third lemma 45.04.07 gives, under the alternative,

S^{*} d N (μ, 1), μ = Cov (S^{*}, Λ) = h \frac{mn}{N} \frac{\int _{0}^{1} φ ^{\circ} φ _{f}^{\circ}}{( \int _{0}^{1} ( φ ^{\circ} ) ^{2} ) ^{1/2}},

with $φ_{f} = - f^{'} / f \circ F^{- 1}$ the alternative's log-derivative score. The limiting power $1 - Φ (z_{1 - α} - μ)$ is maximised over $φ$ , by Cauchy-Schwarz, exactly when $φ^{\circ} \propto φ_{f}^{\circ}$ ^{[Hájek, Šidák & Sen — Theory of Rank Tests (2nd ed.)]}.

Theorem 3 (locally most powerful rank tests and the optimal score). Among all rank tests of the location alternative against the null of randomness, the test maximising the slope of the power function at the null — the locally most powerful rank test — rejects for large values of $S$ with scores

a_{N} (j) = E φ_{f} (U_{(j)}), φ_{f} (u) = - \frac{f ^{'} ( F ^{- 1} ( u ) )}{f ( F ^{- 1} ( u ) )} .

The score is the conditional expectation, given the ranks, of the derivative of the log-likelihood at the null shift; it is the projection of the efficient score onto rank-measurable functions. The canonical specialisations are Wilcoxon scores for the logistic, normal / van der Waerden scores $Φ^{- 1} (j / (N + 1))$ for the Gaussian, and sign / median scores for the Laplace ^{[Hájek & Šidák — Theory of Rank Tests]}.

Theorem 4 (asymptotic relative efficiency and asymptotic optimality). The ARE of the $φ$ -score rank test relative to the $t$ -test against a location alternative with density $f$ is

e (φ, t) = σ_{f}^{2} \frac{( \int _{0}^{1} φ ^{\circ} φ _{f}^{\circ} ) ^{2}}{\int _{0}^{1} ( φ ^{\circ} ) ^{2}},

the ratio of limiting noncentralities of 45.04.05. With the optimal score $φ = φ_{f}$ this equals $σ_{f}^{2} I (f)$ , the product of variance and Fisher information, which is $\geq 1$ with equality only at the Gaussian; the normal-scores test has ARE $\geq 1$ against the $t$ -test for every $f$ (the Chernoff-Savage theorem), and the Wilcoxon-to- $t$ ARE is $3/ π$ at the normal and unbounded for heavy tails ^{[van der Vaart — Asymptotic Statistics]}.

Synthesis. The foundational reason rank-test theory is tractable is that a simple linear rank statistic, though built from the strongly dependent permutation $R$ , is asymptotically linear in the independent scores $φ (F (X_{i}))$ , so its null variance is the without-replacement combinatorial formula and its limit law is the central limit theorem applied to the projection. This is exactly the Hájek-projection method, and it generalises the U-statistic linearisation of 45.05.05 from symmetric kernels to the rank transform; the central insight is that both are the $L^{2}$ -orthogonal projection of a near-linear functional of the empirical measure onto sums of single-coordinate functions, with the dependence relegated to a vanishing-relative-variance remainder. The alternative theory is dual to the null theory: where the null analysis fixes the uniform law on $S_{N}$ , the contiguous-alternative analysis tilts the resulting Gaussian by one covariance through Le Cam's third lemma 45.04.07, and putting these together the local power is the null distribution translated, never recomputed.

The optimal-score theory is the third reading of the same object — the score that maximises the tilt $Cov (S^{*}, Λ)$ is, by Cauchy-Schwarz, the alternative's own log-derivative $φ_{f} = - f^{'} / f \circ F^{- 1}$ , so Wilcoxon, normal, and sign scores are the logistic, Gaussian, and Laplace cases of a single formula, and the asymptotic relative efficiency of 45.04.05 is the squared cosine between the chosen and the optimal score. The bridge from the empirical-process viewpoint of the chapter is that $S$ centred is the empirical measure integrated against $c \cdot φ$ , so the Hájek projection is the rank-test shadow of the Donsker linearisation, and the whole theory is one functional read at the null, at the alternative, and at the optimum.

Full proof set Master

Proposition 1 (exact null mean and variance). For $S = \sum_{i} c_{i} a_{N} (R_{i})$ with $R$ uniform on $S_{N}$ , $E_{0} S = N \overset{c}{ˉ} \overset{a}{ˉ}$ and $Var_{0} (S) = \frac{1}{N - 1} \sum_{i} (c_{i} - \overset{c}{ˉ})^{2} \sum_{j} (a_{N} (j) - \overset{a}{ˉ})^{2}$ .

Proof. Marginal uniformity of each $R_{i}$ gives $E_{0} a_{N} (R_{i}) = N^{- 1} \sum_{j} a_{N} (j) = \overset{a}{ˉ}$ , hence $E_{0} S = \overset{a}{ˉ} \sum_{i} c_{i} = N \overset{c}{ˉ} \overset{a}{ˉ}$ . Centre $c_{i}^{\circ} = c_{i} - \overset{c}{ˉ}$ , $a_{N}^{\circ} (j) = a_{N} (j) - \overset{a}{ˉ}$ . Exchangeability of $(R_{1}, \dots, R_{N})$ makes $Var_{0} (a_{N}^{\circ} (R_{i})) = v := \frac{1}{N} \sum_{j} a_{N}^{\circ} (j)^{2}$ common to all $i$ and $Cov_{0} (a_{N}^{\circ} (R_{i}), a_{N}^{\circ} (R_{k}))$ common to all $i \neq = k$ . Since $\sum_{k} a_{N}^{\circ} (R_{k}) = \sum_{j} a_{N}^{\circ} (j) = 0$ has zero variance, $0 = N v + N (N - 1) Cov_{0}$ , so $Cov_{0} = - v / (N - 1)$ . Then

Var_{0} (S) = i \sum (c_{i}^{\circ})^{2} v - \frac{v}{N - 1} i \neq = k \sum c_{i}^{\circ} c_{k}^{\circ} = v i \sum (c_{i}^{\circ})^{2} + \frac{v}{N - 1} i \sum (c_{i}^{\circ})^{2} = \frac{N v}{N - 1} i \sum (c_{i}^{\circ})^{2},

using $\sum_{i \neq = k} c_{i}^{\circ} c_{k}^{\circ} = (\sum_{i} c_{i}^{\circ})^{2} - \sum_{i} (c_{i}^{\circ})^{2} = - \sum_{i} (c_{i}^{\circ})^{2}$ . Substituting $v = \frac{1}{N} \sum_{j} a_{N}^{\circ} (j)^{2}$ gives the stated formula. $□$

Proposition 2 (Hájek asymptotic linearity). Let scores be generated by a square-integrable $φ$ with $\int_{0}^{1} (φ^{'})^{2} < \infty$ (or $φ$ of bounded variation). Then $S - E_{0} S = \sum_{i} c_{i}^{\circ} φ^{\circ} (F (X_{i})) + R_{N}$ with $E_{0} R_{N}^{2} = o (Var_{0} S)$ .

Proof (load-bearing steps). Write $a_{N} (R_{i}) = φ (R_{i} / (N + 1)) = φ (F_{N} (X_{i}) \cdot \frac{N}{N + 1})$ with $F_{N}$ the pooled empirical c.d.f., and compare with $φ (F (X_{i}))$ . The Hájek projection $\hat{S} = E_{0} S + \sum_{i} (E [S ∣ X_{i}] - E_{0} S)$ is, by the i.i.d. structure, $E_{0} S + \sum_{i} c_{i}^{\circ} ψ_{N} (X_{i})$ where $ψ_{N} (x) = E [a_{N} (R_{i}) ∣ X_{i} = x] - \overset{a}{ˉ}$ . As $N \to \infty$ , $ψ_{N} (x) \to φ^{\circ} (F (x))$ in $L^{2} (F)$ because the conditional rank $R_{i} ∣ X_{i}$ concentrates at $N F (x)$ and $φ$ is square-integrable, so the projection's leading term is $\sum_{i} c_{i}^{\circ} φ^{\circ} (F (X_{i}))$ . The residual $S - \hat{S}$ is orthogonal to all sums of single-coordinate functions (a conditional expectation is an $L^{2}$ -orthogonal projection, as in 45.05.05); its variance is the higher-order empirical-process fluctuation of $φ (F_{N}) - φ (F)$ integrated against $c^{\circ}$ , of order $o (Var_{0} S)$ when $φ$ has the stated smoothness. By Pythagoras $E_{0} (S - \hat{S})^{2} = Var_{0} S - Var_{0} \hat{S} = o (Var_{0} S)$ . Setting $R_{N} = (S - \hat{S}) + (\hat{S} - E_{0} S - \sum_{i} c_{i}^{\circ} φ^{\circ} (F (X_{i})))$ collects both vanishing pieces. $□$

Proposition 3 (asymptotic normality under the Noether condition). If $φ$ is non-constant and square-integrable and $max_{i} (c_{i}^{\circ})^{2} / \sum_{i} (c_{i}^{\circ})^{2} \to 0$ , then $(S - E_{0} S) / Var_{0} S d N (0, 1)$ under the null.

Proof. By Proposition 2, $(S - E_{0} S) / Var_{0} S = (\sum_{i} c_{i}^{\circ} φ^{\circ} (F (X_{i}))) / Var_{0} S + o_{P} (1)$ . The summands $ξ_{i} = c_{i}^{\circ} φ^{\circ} (F (X_{i}))$ are independent (the $F (X_{i})$ are i.i.d. uniform under the null), mean zero, with $\sum_{i} Var ξ_{i} = (\int (φ^{\circ})^{2}) \sum_{i} (c_{i}^{\circ})^{2} \sim Var_{0} S$ . The Lindeberg ratio is controlled by $max_{i} (c_{i}^{\circ})^{2} / \sum_{i} (c_{i}^{\circ})^{2} \to 0$ (the Noether condition), since $φ^{\circ} (F (X_{i}))$ has a fixed finite variance and the weights' maximal relative contribution vanishes. The Lindeberg-Feller central limit theorem (the version used in 45.05.05) gives a standard normal limit for the leading sum, and Slutsky's lemma 45.04.07 absorbs the $o_{P} (1)$ remainder. $□$

Proposition 4 (contiguous-alternative limit and local power). In the two-sample location-shift LAN family with shift $h / N$ , under the contiguous alternative $S^{*} = (S - E_{0} S) / Var_{0} S d N (μ, 1)$ with $μ = h mn / N ρ_{φ, φ_{f}} (\int (φ^{\circ})^{2})^{1/2}$ and $ρ$ the standardised inner product of $φ^{\circ}, φ_{f}^{\circ}$ ; the limiting power is $1 - Φ (z_{1 - α} - μ)$ .

Proof. The LAN log-ratio satisfies $Λ_{N} d N (- \frac{1}{2} σ^{2}, σ^{2})$ under the null 45.04.07, and by Proposition 2 the pair $(S^{*}, Λ_{N})$ converges jointly to a bivariate Gaussian, since both are asymptotically linear in the i.i.d. scores: $S^{*}$ in $φ^{\circ} (F (X_{i}))$ and $Λ_{N}$ in $φ_{f}^{\circ} (F (X_{i}))$ (the latter by the LAN expansion, whose efficient score is the log-derivative $φ_{f}$ ). Their limiting covariance is $c = Cov (S^{*}, Λ) = h mn / N \int_{0}^{1} φ^{\circ} φ_{f}^{\circ} / (\int (φ^{\circ})^{2})^{1/2}$ by the bilinearity of covariance on the projections. Le Cam's third lemma 45.04.07 tilts the null $N (0, 1)$ law of $S^{*}$ to $N (c, 1)$ under the contiguous alternative, so $μ = c$ , and the one-sided power is $P (N (μ, 1) > z_{1 - α}) = 1 - Φ (z_{1 - α} - μ)$ . $□$

Proposition 5 (optimal score and ARE). The score $φ$ maximising the local power $μ$ over the unit sphere $\int (φ^{\circ})^{2} = 1$ is $φ^{\circ} \propto φ_{f}^{\circ}$ , giving the locally most powerful rank test; the ARE relative to the $t$ -test is $e (φ, t) = σ_{f}^{2} (\int φ^{\circ} φ_{f}^{\circ})^{2} / \int (φ^{\circ})^{2}$ , maximised at $σ_{f}^{2} I (f)$ .

Proof. From Proposition 4, $μ \propto \int_{0}^{1} φ^{\circ} φ_{f}^{\circ} / (\int (φ^{\circ})^{2})^{1/2}$ . By the Cauchy-Schwarz inequality $\int φ^{\circ} φ_{f}^{\circ} \leq (\int (φ^{\circ})^{2})^{1/2} (\int (φ_{f}^{\circ})^{2})^{1/2}$ with equality iff $φ^{\circ} \propto φ_{f}^{\circ}$ ; hence the LMP score is $φ_{f} = - f^{'} / f \circ F^{- 1}$ , and its smoothed form $a_{N} (j) = E φ_{f} (U_{(j)})$ is the rank-measurable projection of the efficient score. The ARE is the squared ratio of the two tests' limiting noncentralities 45.04.05: the $φ$ -rank test has noncentrality $\propto \int φ^{\circ} φ_{f}^{\circ} / (\int (φ^{\circ})^{2})^{1/2}$ and the $t$ -test has noncentrality $\propto 1/ σ_{f}$ , giving $e (φ, t) = σ_{f}^{2} (\int φ^{\circ} φ_{f}^{\circ})^{2} / \int (φ^{\circ})^{2}$ . At $φ = φ_{f}$ this is $σ_{f}^{2} \int (φ_{f}^{\circ})^{2} = σ_{f}^{2} I (f)$ , the variance-times-Fisher-information product, $\geq 1$ by Cauchy-Schwarz with equality only at the Gaussian. $□$

Connections Master

The Hájek-projection method that linearises the rank statistic is the same device proved for symmetric kernels in U-statistics and their asymptotics 45.05.05: there a non-degenerate U-statistic is dominated in $L^{2}$ by its projection onto sums of single-coordinate conditional expectations, and here the simple linear rank statistic is dominated by its projection onto the independent scores $φ (F (X_{i}))$ ; both reduce a dependent statistic to an i.i.d. average whose central limit theorem furnishes the limit law, and the orthogonality of the residual is the $L^{2}$ -projection identity common to both.

Contiguity and Le Cam's three lemmas 45.04.07 supply the entire alternative-distribution theory: the location-shift alternatives are contiguous to the null of randomness, the joint null limit of the standardised rank statistic with the LAN log-likelihood ratio is bivariate Gaussian, and Le Cam's third lemma tilts the null normal law of the statistic into its alternative form by the single covariance $Cov (S^{*}, Λ)$ , which is exactly the local-power noncentrality and the quantity the optimal score maximises.

Asymptotic relative efficiency 45.04.05 is the metric that ranks rank tests: the ARE of the $φ$ -score test relative to the $t$ -test is the squared cosine between the chosen score $φ$ and the optimal score $φ_{f}$ times the variance-Fisher-information factor, so the Pitman-efficiency comparisons computed there — Wilcoxon's $3/ π$ at the Gaussian, the unbounded gain at heavy tails, the Chernoff-Savage lower bound for normal scores — are corollaries of the optimal-score theorem of this unit.

The empirical distribution function and the Glivenko-Cantelli theorem 45.05.02 underlie the projection step: the rank $R_{i} = N F_{N} (X_{i})$ is $N$ times the empirical c.d.f. at $X_{i}$ , and the replacement of $φ (R_{i} / (N + 1))$ by $φ (F (X_{i}))$ is controlled by the uniform closeness of $F_{N}$ to $F$ , so the asymptotic linearity of the rank statistic is the empirical-process linearisation specialised to the rank transform.

The Donsker classes and the empirical process 45.05.03 generalise the fixed-score rank statistic to rank processes indexed by the score function or by a threshold: the leading term $\sum_{i} c_{i}^{\circ} φ (F (X_{i}))$ is the empirical measure integrated against $c^{\circ} φ$ , and the higher-order remainder is governed by the modulus of continuity of the empirical process, so the weak convergence of rank statistics is a corollary of the Donsker theory there.

Historical & philosophical context Master

Rank tests entered statistics through Frank Wilcoxon's 1945 note proposing the rank-sum and signed-rank procedures, and through the equivalent Mann-Whitney form of 1947; the idea of replacing measurements by ranks to obtain a distribution-free test was already implicit in the sign test and in Spearman's rank correlation of 1904. The decisive theoretical advance was the recognition that, under the hypothesis of randomness, the rank vector is uniform on the symmetric group, so any rank statistic has an exactly computable, distribution-free null law.

The general asymptotic theory is the achievement of Jaroslav Hájek and Zbyněk Šidák, whose 1967 monograph Theory of Rank Tests ^{[Hájek & Šidák — Theory of Rank Tests]} systematised the exact null distribution theory, the projection lemma for asymptotic normality, the use of contiguity for local power, and the theory of locally most powerful and asymptotically optimal scores. Hájek's 1968 Annals of Mathematical Statistics paper on the asymptotic normality of simple linear rank statistics under alternatives ^{[Hájek, J. — Asymptotic normality of simple linear rank statistics under alternatives]} isolated the projection of a rank statistic onto sums of independent terms as the key device, the same projection later seen to drive U-statistic asymptotics.

The optimal-score theory — that the locally most powerful rank score is the log-derivative $- f^{'} / f$ of the assumed density pushed through the quantile transform, yielding Wilcoxon scores for the logistic, van der Waerden's normal scores for the Gaussian, and sign scores for the Laplace — was developed by Hájek and Šidák building on the score-function programme of Erich Lehmann and others; the second edition with Pranab K. Sen ^{[Hájek, Šidák & Sen — Theory of Rank Tests (2nd ed.)]} is the standard modern reference. Herman Chernoff and I. Richard Savage proved in 1958 that the normal-scores test is at least as efficient as the $t$ -test against every alternative, and the synthesis within the empirical-process and Le Cam frameworks is due to van der Vaart ^{[van der Vaart — Asymptotic Statistics]}.

Bibliography Master

@book{HajekSidak1967,
  author    = {H\'ajek, Jaroslav and \v{S}id\'ak, Zbyn\v{e}k},
  title     = {Theory of Rank Tests},
  publisher = {Academic Press},
  address   = {New York},
  year      = {1967}
}

@book{HajekSidakSen1999,
  author    = {H\'ajek, Jaroslav and \v{S}id\'ak, Zbyn\v{e}k and Sen, Pranab K.},
  title     = {Theory of Rank Tests},
  edition   = {2},
  publisher = {Academic Press},
  address   = {San Diego},
  year      = {1999}
}

@article{Hajek1968,
  author  = {H\'ajek, Jaroslav},
  title   = {Asymptotic normality of simple linear rank statistics under alternatives},
  journal = {Annals of Mathematical Statistics},
  volume  = {39},
  number  = {2},
  year    = {1968},
  pages   = {325--346}
}

@book{vanderVaart1998,
  author    = {van der Vaart, Aad W.},
  title     = {Asymptotic Statistics},
  publisher = {Cambridge University Press},
  series    = {Cambridge Series in Statistical and Probabilistic Mathematics},
  year      = {1998}
}

@book{Lehmann1975,
  author    = {Lehmann, Erich L.},
  title     = {Nonparametrics: Statistical Methods Based on Ranks},
  publisher = {Holden-Day},
  address   = {San Francisco},
  year      = {1975}
}

@article{Wilcoxon1945,
  author  = {Wilcoxon, Frank},
  title   = {Individual comparisons by ranking methods},
  journal = {Biometrics Bulletin},
  volume  = {1},
  number  = {6},
  year    = {1945},
  pages   = {80--83}
}

@article{ChernoffSavage1958,
  author  = {Chernoff, Herman and Savage, I. Richard},
  title   = {Asymptotic normality and efficiency of certain nonparametric test statistics},
  journal = {Annals of Mathematical Statistics},
  volume  = {29},
  number  = {4},
  year    = {1958},
  pages   = {972--994}
}

Prerequisites

45.05.05
45.04.07

Tier anchors

beginner: Hollander, Wolfe & Chicken 2014 Nonparametric Statistical Methods 3e (Wiley) Ch. 2-4 (ranks, the Wilcoxon rank-sum statistic, distribution-free tests built from ranks); Lehmann 1975 Nonparametrics: Statistical Methods Based on Ranks (Holden-Day) Ch. 1 (the rank transform and why it frees inference from the underlying shape)
intermediate: van der Vaart 1998 Asymptotic Statistics (Cambridge) Ch. 13 (rank, signed-rank statistics; the null mean/variance of a linear rank statistic, asymptotic normality via the projection method, the limit under contiguous alternatives, and locally most powerful scores); Hájek & Šidák 1967 Theory of Rank Tests (Academic Press) Ch. II-V (exact null distribution theory, the projection lemma, optimal scores)
master: Hájek & Šidák 1967 Theory of Rank Tests (Academic Press) Ch. II (rank distribution theory), Ch. V (the projection / Hájek-projection lemma and asymptotic normality under alternatives), Ch. VI (contiguity), Ch. VII (locally most powerful rank tests and asymptotically optimal scores); Hájek, Šidák & Sen 1999 Theory of Rank Tests 2e (Academic Press); van der Vaart 1998 Asymptotic Statistics (Cambridge) Ch. 13-14 (rank tests and their asymptotic efficiency); Hájek 1968 Asymptotic normality of simple linear rank statistics under alternatives (Ann. Math. Statist. 39)

References

van der Vaart — Asymptotic Statistics · Cambridge University Press, Cambridge Series in Statistical and Probabilistic Mathematics, 1998; Ch. 13 (rank and signed-rank statistics: distribution-freeness under the null, the exact null mean and variance of a simple linear rank statistic, asymptotic normality through the Hájek projection, the contiguous-alternative limit via Le Cam's third lemma, and locally most powerful rank tests with their optimal score functions)
Hájek, J. & Šidák, Z. — Theory of Rank Tests · Academic Press, New York, 1967; Ch. II (the distribution of ranks and order statistics under the hypothesis of randomness), Ch. V (the projection lemma and asymptotic normality of simple linear rank statistics under contiguous alternatives), Ch. VI (contiguity), Ch. VII (locally most powerful rank tests and asymptotically optimal scores: Wilcoxon, van der Waerden / normal, median / sign scores)
Hájek, J. — Asymptotic normality of simple linear rank statistics under alternatives · Annals of Mathematical Statistics 39 (1968), 325-346 (the projection of a simple linear rank statistic onto sums of independent terms and the resulting asymptotic normality under fixed and contiguous alternatives)
Hájek, Šidák & Sen — Theory of Rank Tests (2nd ed.) · Academic Press, San Diego, 1999; Ch. II-VII (modern treatment of rank distribution theory, the projection method, contiguity, and locally most powerful and asymptotically optimal rank tests)
Lehmann, E. L. — Nonparametrics: Statistical Methods Based on Ranks · Holden-Day, San Francisco, 1975; Ch. 1-5 (the rank transform, the Wilcoxon and normal-scores tests, two-sample and regression rank procedures, and the efficiency of rank tests)

Estimated time

beginner: 20m
intermediate: 55m
master: 95m