Item response theory and the Rasch model: latent-trait measurement
Anchor (Master): Rasch 1960 Probabilistic Models; Lord 1980 Applications of IRT; Birnbaum 1968 (2PL/3PL); Samejima 1969 (graded response); Embretson-Reise 2013 modern treatment
Intuition Beginner
Classical test theory, the framework of the previous unit, treats a test as a single observed score plus a single lump of error. Item response theory turns the lens around: it models each item on its own. The guiding question shifts from "what is the person's true total score?" to "what is the probability that this particular person answers this particular item correctly?"
The engine behind that shift is a hidden quantity called the latent trait, written (theta). A person's is their standing on whatever the test measures — reasoning skill, reading fluency, anxiety level. Each item carries a difficulty . The probability of a correct answer rises smoothly as the gap between ability and difficulty grows.
The shape of that rise is an S-shaped curve. When ability sits far below difficulty, the probability of success is near zero. When ability sits far above, the probability is near one. Right at the crossing point — where ability equals difficulty — the probability is exactly one half, a coin flip. This S-curve is the item characteristic curve, and it is the basic object of the theory.
Why model the item rather than the total score? Because items differ, and a person's raw score depends on which items they happened to see. Item response theory places every person and every item on one shared scale, so that a comparison of two people does not depend on which items were used, and a comparison of two items does not depend on which people took them. That invariance is something classical test theory cannot offer.
Visual Beginner
The single curve on the left shows that an item measures most sharply near its own difficulty , where the slope is steepest and each unit of ability buys the largest change in success probability. The family on the right shows that spreading item difficulties across the ability range lets a test measure precisely at every point of the continuum rather than only at one.
Worked example Beginner
A reasoning item has difficulty . Two students attempt it: student A has ability , student B has ability . The Rasch rule for the probability of a correct answer is
The quantity is the gap between the student's ability and the item's difficulty. A positive gap favours the student; a negative gap favours the item.
For student A the gap is . For student B the gap is . The arithmetic for each is:
Student A (gap = +0.5): e^0.5 = 1.65 -> P = 1.65 / (1 + 1.65) = 1.65 / 2.65 = 0.62
Student B (gap = -0.5): e^-0.5 = 0.61 -> P = 0.61 / (1 + 0.61) = 0.61 / 1.61 = 0.38The stronger student has a 62% chance of success and the weaker one a 38% chance. Notice that both probabilities stay well away from 0 and 1: even the weaker student has a real chance of being right, and even the stronger student can miss it. That is the forgiving shape of the logistic curve — never a cliff.
The shared unit of and is the logit. A difference of one logit between ability and difficulty multiplies the odds of a correct answer by , so logits are the natural currency in which ability and difficulty are compared.
Check your understanding Beginner
Formal definition Intermediate+
Item response theory (IRT) [Rasch1960] is a family of models for the probability that person with latent ability gives a designated response to item with parameters . For the dichotomous case (correct/incorrect, coded ), the models are nested by adding one parameter at a time.
1PL — the Rasch model. Each item carries a single difficulty , and [Rasch1960]
The equivalent logit form is the additive identity that defines the model:
Ability and difficulty share the logit scale, so an item is harder exactly when is larger, and a person is more able exactly when is larger. The subtraction is the whole model: the log-odds of success is the person's location minus the item's location.
2PL — adding discrimination. [Birnbaum1968] The two-parameter logistic model introduces a slope parameter , the discrimination, so that
A large makes the item characteristic curve steep (the item sharply separates people just above its difficulty from those just below); a small makes it shallow. The Rasch model is the special case .
3PL — adding guessing. A lower asymptote accounts for the chance of answering a multiple-choice item correctly by guessing, giving
The curve no longer falls to zero as ; it approaches , the success probability of a pure guess. Each added parameter buys flexibility at the cost of more data needed to estimate it stably.
Polytomous extensions. For ordered-category items (Likert scales, partial credit), Samejima's graded response model [Samejima1969] and the partial-credit and rating-scale models generalise the logistic link to cumulative boundaries between adjacent categories. The taxonomy of Thissen and Steinberg [Thissen1986] unifies these under one family.
Information and standard error. The item information function measures how much each item reduces uncertainty about at each point of the ability continuum. For the 2PL,
and for the Rasch model this reduces to since . The test information is the sum over items,
and the asymptotic standard error of the ability estimate is
Information is local: a test can be precise near the average ability and imprecise in the tails, which is why well-designed tests spread item difficulties across the range of interest. The standard error is largest where information is smallest, so precision varies with ability rather than being a single number as in classical test theory.
Counterexamples to common slips Intermediate+
- Equal slopes are a model assumption, not an empirical fact. The Rasch model requires ; forcing this on data where items genuinely differ in discrimination misfits the curve. The 2PL relaxes this, at the cost of estimating one more parameter per item.
- Information is local, reliability is global. A single CTT reliability coefficient summarises the whole test, but varies with ability. Two tests with identical reliability can have very different precision profiles.
- The latent scale is arbitrary up to a linear transform. IRT fixes only differences ; the origin and unit of the logit scale are set by a convention (mean , or anchoring item difficulties).
- Guessing breaks specific objectivity. The 3PL lower asymptote reintroduces an item-specific term that prevents the clean cancellation ; the strong invariance results hold only for the 1PL.
Key model Intermediate+
The Rasch model is the central object of this unit, and its defining strength is specific objectivity. Because the logit of success is the additive identity , the comparison of any two persons on a common item depends only on their abilities:
which is free of the item difficulty . Symmetrically, the comparison of two items on a common person depends only on their difficulties. This is a strong invariance claim: the ordering of people is independent of the calibre of items used, and the ordering of items is independent of the people who took them — provided the items fit the model. Classical test theory cannot match this, because a raw score there is bound to the specific items on the test.
The price of specific objectivity is the restriction : all items must discriminate equally well. The 2PL and 3PL buy flexibility by surrendering the clean additive form, and with it the exact cancellation that specific objectivity requires. Choosing between the 1PL and its generalisations is a substantive modelling decision settled by fit statistics (likelihood-ratio tests, , item-fit residuals) rather than by fiat.
Estimation. Three routes recover and the item parameters from a response matrix. Joint maximum likelihood (JML) estimates all person and item parameters simultaneously; it is plain in concept but biased for short tests because the number of parameters grows with the number of people. Conditional maximum likelihood exploits the Rasch property that the raw score is a sufficient statistic for , conditioning it out and estimating item parameters alone — consistent and unbiased for the 1PL [Rasch1960]. Marginal maximum likelihood (MML) assumes an ability distribution across the population, integrates out, and estimates item parameters from the marginal likelihood, typically via the EM algorithm [Lord1980]. Person abilities are then estimated in a second step by maximum likelihood or Bayes' expected a posteriori. MML is the modern default because it handles sparse data and generalises cleanly to 2PL and 3PL.
Worked information example. Three Rasch items have difficulties . At ability the success probabilities are , , , so the item informations are . The test information is , giving logits. A 3-item test is blunt; a 30-item test spreading difficulties across the same range would lift by roughly an order of magnitude and cut the standard error in proportion.
Bridge. The Rasch model builds toward the information-theoretic view of precision that classical test theory could only approximate with a single reliability number, and appears again in the factor-analysis companion as the item loading, since a 2PL discrimination parameter is a rescaled factor loading under linearisation. The foundational reason the logistic link is preferred is that it places ability and difficulty additively on the logit scale, so item and person share one continuum; this is exactly the condition that specific objectivity requires. Putting these together, the item characteristic curve is the basic measuring unit, and the bridge is that test information converts a bank of curves into a local standard error , replacing CTT's constant standard error of measurement with a precision profile that varies with ability.
Exercises Intermediate+
Advanced results Master
The Rasch model and its generalisations are descriptive once the parameters are in hand; the machinery that puts them there, and that polices their fairness across groups, is the substance of advanced IRT. The results below record how estimation, item fairness, and adaptive testing are actually carried out.
Estimation: joint, conditional, and marginal likelihood
Joint maximum likelihood (JML) writes the likelihood of the full response matrix as a product over persons and items and maximises over every and every at once. Its defect is the incidental-parameters problem: when the number of people grows while each person's number of items stays fixed, the item-parameter estimates are inconsistent, because each person's noisy contaminates the item estimates. JML is intuitive but biased for the test lengths typical of educational measurement.
Conditional maximum likelihood (CML) is available only for the Rasch family. The key fact is that the raw score is a sufficient statistic for , so conditioning on eliminates from the likelihood entirely. The conditional likelihood of the response pattern given the score depends only on the item difficulties, which are then estimated consistently. The cost is that CML does not generalise to the 2PL or 3PL, where no sufficient statistic for exists.
Marginal maximum likelihood (MML) [Lord1980] is the modern default. It posits a population distribution for ability (typically normal with free mean and variance), writes the marginal likelihood of each response pattern as , and maximises over the item parameters and the distribution's moments. The integral has no closed form and is evaluated by the EM algorithm: the E-step computes the expected sufficient statistics given current parameters, and the M-step updates the parameters. Bock and Aitkin's 1981 implementation remains the workhorse. Person abilities are recovered afterwards by maximum likelihood or by the expected a posteriori (EAP) estimate, which shrinks toward the population mean and is preferred for short tests. MML handles the 2PL and 3PL naturally and underpins every modern item-bank calibration.
Differential item functioning
Differential item functioning (DIF) is the operational definition of item unfairness: an item shows DIF when examinees of equal ability but from different demographic groups have different success probabilities on it. DIF is item-level non-invariance, distinct from a real group difference in the trait, and detecting it is a legal and ethical prerequisite for high-stakes testing.
The Mantel-Haenszel procedure [Holland1988] is the simplest robust detector. Examinees are stratified into narrow ability bands (typically by total score), and within each band the odds of a correct response for the focal group are compared to those for the reference group. The common odds ratio pools across strata; means no DIF, and the transformation places the magnitude on the ETS delta scale used to flag items (A/B/C categories). The Mantel-Haenszel test is cheap and assumes uniform DIF (the same bias across the ability range).
The likelihood-ratio test is the more flexible alternative. It fits two IRT models — one with a common item parameter for both groups, one with a group-specific parameter — and compares the fit via likelihood. Uniform and non-uniform DIF (where discrimination itself differs across groups) can be tested by nesting models appropriately. The likelihood-ratio framework generalises cleanly to the 2PL and 3PL and is the default for publishable DIF analyses.
Specific objectivity and the Rasch justification
Georg Rasch [Rasch1960] regarded specific objectivity not as a convenient by-product but as the defining criterion of measurement. His argument: a comparison of two objects (people, or items) deserves to be called a measurement only if it yields the same result regardless of which instruments (items, or people) are used within a class. The additive logit form is the unique link (up to scale) for which this holds for binary responses, and Rasch took this as a reason to prefer the 1PL even when the 2PL fits marginally better. The position is contested — the 2PL camp argues that forcing equal discrimination sacrifices descriptive truth for an invariance guarantee the data do not support — but it marks a genuine philosophical fork: whether measurement is defined by the model's invariance properties or by its fit to the data.
Comparison with classical test theory
IRT and CTT answer the same question — how precisely does this instrument measure? — at different resolution. CTT operates on the total score and reports a single reliability coefficient and a single standard error of measurement; IRT operates on each item and reports a precision profile that varies with ability. A CTT score is bound to the specific items on the test, so comparing scores across different forms requires equating; IRT places person and item on a common scale, so scores are comparable across forms by construction. CTT's reliability is population-relative (it depends on the true-score variance in the sample), whereas IRT item parameters are invariant across populations once the scale is fixed. The cost of IRT is larger sample sizes for stable calibration and a parametric model whose assumptions must be checked. For low-stakes group research CTT is often sufficient; for high-stakes individual decisions and adaptive testing IRT is the standard.
Applications: computerised adaptive testing
Computerised adaptive testing (CAT) is the practical payoff of item information. A CAT engine maintains a calibrated item bank, begins from a provisional ability estimate, and at each step selects the item that maximises information at the current , administers it, updates by maximum likelihood or Bayes, and stops when the cumulative information crosses a target (equivalently, when falls below a threshold). Because each examinee receives items targeted near their own ability, a CAT achieves the precision of a long fixed test with roughly half the items — the GRE, GMAT, and NATO's military aptitude batteries are all CAT implementations built on 3PL item banks. The same machinery underpins the scaling of the SAT and the equating of test forms across administrations: IRT's common logit scale is what allows scores from different item sets to be placed on one reported scale.
Synthesis. Putting these together, the Rasch model, its 2PL/3PL generalisations, and the information apparatus form a single measurement engine: the logistic link is the foundational reason item-level probabilities admit a closed-form Fisher information, this is exactly what turns a bank of items into a precision-weighted measuring instrument, and the central insight is that precision varies with ability rather than being a single number as in classical theory. Specific objectivity generalises the parallel-tests idea of the sibling unit into a statement that comparisons are invariant to the calibre of items used, and the bridge is that estimation (joint, conditional, marginal) and DIF testing together operationalise the fairness claims that reliability alone cannot reach. The pattern recurs whenever a latent variable must be measured: the same information-versus-standard-error duality appears again in the standard error of measurement of classical test theory, and item response theory is dual to factor analysis in that the discrimination parameter and the factor loading are rescaled versions of one another under the linearised link.
Full proof set Master
Proposition (Fisher information and standard error under the 1PL Rasch model). Let be the responses of a fixed person with latent ability to Rasch items with difficulties , modelled as independent Bernoulli variables with . Then (a) the Fisher information about carried by item is ; (b) the test information is ; and (c) the large-sample standard error of the maximum-likelihood estimator satisfies .
Proof. For (a), write the log-likelihood of a single response as
The logistic identity gives , so . Differentiating the two log-probabilities against yields and respectively, where . Hence the score is
For the logistic curve the derivative is , which cancels the denominator and reduces the score to the clean form
The Fisher information is the variance of the score (its mean is zero, since ):
using that the variance of a Bernoulli variable with success probability is . Equivalently, , which equals by the same logistic derivative.
For (b), independence of the items makes the total log-likelihood a sum . Scores of independent components add, so the Fisher information is additive:
For (c), the maximum-likelihood estimator is asymptotically normal about the true with variance given by the inverse Fisher information — the Cramér-Rao lower bound, attained in the large-sample limit. Because the total information is , the asymptotic variance is , and taking square roots gives
The proposition is the formal content of the statement that a test measures precisely where it is informative. Because peaks at , i.e. at , each item contributes most where the person sits right at its difficulty; spreading item difficulties spreads the peaks and raises across the range, lowering the standard error there. The identity converts the abstract information curve into a concrete confidence band on a person's ability, and it is the reason adaptive testing targets items near the current .
Connections Master
Psychometrics foundations
29.13.01. This unit deepens the item-response-theory sketch in the foundations sibling: where that unit surveyed IRT as one of three pillars alongside generalisability theory and factor analysis, the present unit develops the formal machinery (estimation, information, DIF, specific objectivity) that the survey only named. Every claim here presupposes the reliability-validity distinction and the classical-test-theory decomposition fixed in the foundational unit.Classical test theory and factor analysis
29.13.02. IRT is the item-level counterpart to the test-level theory of the CTT companion. The CTT standard error of measurement is a single number; the IRT standard error is its ability-varying generalisation. The 2PL discrimination parameter is a rescaled factor loading under linearisation, so the two units describe the same latent-variable decomposition at different resolution.Cognition and intelligence
29.05.01. Every large-scale ability test — the GRE, the SAT, the military aptitude batteries, the IQ assessments discussed in the intelligence unit — is calibrated and scaled by the IRT models of this unit. Claims about the structure of , about test-retest reliability of an IQ score, and about fairness of selection testing all reduce to item-bank calibration, test information, and DIF analyses defined here.
Historical & philosophical context Master
Item response theory was founded by the Danish mathematician Georg Rasch, who developed the one-parameter model in the 1950s for the Danish Institute for Educational Research to analyse reading-attainment tests [Rasch1960]. Rasch's Probabilistic Models for Some Intelligence and Attainment Tests (1960) introduced not only the model that bears his name but the specific-objectivity criterion he took to be the definition of measurement itself: a comparison deserves to be called measurement only if it is invariant to the instruments used. This philosophical commitment — that the model's invariance properties, not its fit to data, define measurement — set the Rasch tradition apart from the broader IRT community and remains a live fault-line in psychometrics.
The generalisation to multi-parameter models is due to Allan Birnbaum, whose chapters in Lord and Novick's 1968 Statistical Theories of Mental Test Scores introduced the 2PL and 3PL by adding discrimination and guessing parameters [Birnbaum1968]. Frederick Lord then built the modern estimation and application apparatus: his Applications of Item Response Theory to Practical Testing Problems (1980) [Lord1980] developed marginal maximum likelihood, the EM-based calibration that still underpins item-bank estimation, and the first operational computerised adaptive testing engine for the GRE. Fumiko Samejima's 1969 graded response model [Samejima1969] extended the framework from dichotomous to ordered-category items, opening IRT to Likert-scale personality and attitude measurement.
The modern synthesis — treating IRT, CTT, and factor analysis as three parameterisations of one latent-variable decomposition — was consolidated by Embretson and Reise's Item Response Theory for Psychologists (2000, 2nd edition 2013) [Embretson2000], which made the apparatus accessible beyond the educational-measurement community. The 1980s and 1990s added the differential-item-functioning apparatus (Holland and Thayer's Mantel-Haenszel procedure, 1988) [Holland1988] and the Thissen-Steinberg taxonomy of item response models [Thissen1986], both of which turned fairness testing from an aspiration into a routine statistical procedure. The continuing philosophical tension — between those who follow Rasch in restricting measurement to invariant models and those who prefer best-fitting multi-parameter models — is not a technical dispute to be settled but a standing reminder that the choice of model embodies a substantive position about what measurement is for.
Bibliography Master
@book{rasch1960models,
author = {Rasch, Georg},
title = {Probabilistic Models for Some Intelligence and Attainment Tests},
publisher = {Danish Institute for Educational Research},
year = {1960},
note = {Reprinted by University of Chicago Press, 1980}
}
@book{lord1980applications,
author = {Lord, Frederic M.},
title = {Applications of Item Response Theory to Practical Testing Problems},
publisher = {Lawrence Erlbaum Associates},
year = {1980}
}
@incollection{birnbaum1968latent,
author = {Birnbaum, Allan},
title = {Some Latent Trait Models and Their Use in Inferring an Examinee's Ability},
booktitle = {Statistical Theories of Mental Test Scores},
editor = {Lord, Frederic M. and Novick, Melvin R.},
publisher = {Addison-Wesley},
year = {1968},
note = {Chapters 17--20}
}
@techreport{samejima1969estimation,
author = {Samejima, Fumiko},
title = {Estimation of Latent Ability Using a Response Pattern of Graded Scores},
institution = {Psychometric Society},
type = {Psychometric Monograph},
number = {17},
year = {1969}
}
@book{embretsonreise2000,
author = {Embretson, Susan E. and Reise, Steven P.},
title = {Item Response Theory for Psychologists},
publisher = {Lawrence Erlbaum Associates},
year = {2000},
note = {2nd edition, Routledge, 2013}
}
@book{deboeckwilson2004,
author = {De Boeck, Paul and Wilson, Mark},
title = {Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach},
publisher = {Springer},
year = {2004}
}
@article{hollandthayer1988,
author = {Holland, Paul W. and Thayer, Dorothy T.},
title = {An Alternative to the Mantel-Haenszel Procedure},
journal = {Journal of Educational Statistics},
volume = {13},
number = {1},
pages = {27--49},
year = {1988}
}
@article{thissensteinberg1986,
author = {Thissen, David and Steinberg, Lynne},
title = {A Taxonomy of Item Response Models},
journal = {Psychometrika},
volume = {51},
number = {4},
pages = {567--577},
year = {1986}
}
@article{bockaitkin1981,
author = {Bock, R. Darrell and Aitkin, Murray},
title = {Marginal Maximum Likelihood Estimation of Item Parameters: Application of an {EM} Algorithm},
journal = {Psychometrika},
volume = {46},
number = {4},
pages = {443--459},
year = {1981}
}
@book{lordnovick1968,
author = {Lord, Frederic M. and Novick, Melvin R.},
title = {Statistical Theories of Mental Test Scores},
publisher = {Addison-Wesley},
year = {1968}
}