Econometrics — identification, estimation, and inference
Anchor (Master): Wooldridge Econometric Analysis of Cross Section and Panel Data (MIT Press); Hamilton Time Series Analysis (Princeton); Angrist & Pischke Mostly Harmless Econometrics (Princeton) — full asymptotic proofs and the identification agenda
Intuition Beginner
Econometrics uses data and statistics to measure economic relationships and to test economic theory against evidence. Its central problem is older and harder than drawing a best-fit line: separating causation from correlation in data we did not produce through a controlled experiment.
Consider the observation that towns with more doctors also record higher death rates. A naive reader concludes doctors cause death. The econometrician asks whether a third factor — an older or sicker population — drives both, so that more doctors and more deaths are joint symptoms of the same underlying condition.
This is the endogeneity problem: the variable we treat as a cause is tangled with other factors that also shape the outcome. Economists rarely run controlled experiments, so most of the discipline's craft is about recovering causal effects from observational data using assumptions, instruments, and careful research design.
Econometrics gives that craft a formal language: a model of where the data came from, an estimator that turns data into a number, and a measure of how much trust that number deserves.
Visual Beginner
A scatter of data points with a straight line drawn through them. The slope of the line measures how much the outcome moves when moves by one unit. The vertical gaps from each point to the line are the residuals — the part of the line does not explain.
The ordinary least squares line is the one that makes the sum of squared residuals as small as possible.
Worked example Beginner
Three observations link years of education () to hourly wage (): , , . We fit the line by ordinary least squares.
Step 1. Averages: and .
Step 2. Build the deviation table. The slope equals the sum of the cross-products divided by the sum of the squared -deviations.
| 1 | 2 | -1 | -1.333 | 1.333 | 1 |
| 2 | 3 | 0 | -0.333 | 0.000 | 0 |
| 3 | 5 | 1 | 1.667 | 1.667 | 1 |
| sum | 3.000 | 2 |
Step 3. Slope: . Each extra year of education is associated with 1.5 more dollars of hourly wage.
Step 4. Intercept: .
The fitted line is . At it predicts , which sits close to the observed wage of .
Check your understanding Beginner
Formal definition Intermediate+
The population linear regression model writes the outcome as a linear function of regressors plus an unobserved error. For observation , with a row containing the regressors and a constant,
where is the parameter vector. In matrix form, stacking rows, with . The OLS estimator minimises the sum of squared residuals,
provided is invertible (equivalently, has full column rank). The fitted values are and the residuals are .
The classical Gauss-Markov assumptions, in matrix form, are:
- MLR.1 (linearity in parameters). .
- MLR.2 (full rank). has full column rank — no perfect multicollinearity.
- MLR.3 (strict exogeneity). for every .
- MLR.4 (spherical errors). — homoscedastic and uncorrelated errors.
- MLR.5 (normality, optional). , which enables exact finite-sample and inference.
Under MLR.1–MLR.3 the OLS estimator is conditionally unbiased: . Adding MLR.4 yields the Gauss-Markov theorem — OLS is the best linear unbiased estimator [Stock Watson Ch 4]. The error variance is estimated unbiasedly by .
Endogeneity is the violation of MLR.3: for some regressor . Three classic sources are omitted variables (a confounder in ), measurement error in a regressor, and simultaneity (a regressor jointly determined with ). Each makes OLS inconsistent for .
Geometry: partialling out (Frisch–Waugh–Lovell)
Partition conformably with . The Frisch–Waugh–Lovell theorem states that the OLS estimate from the full regression of on and equals the estimate from regressing the residual of on against the residual of on :
Geometrically, OLS is the orthogonal projection of onto the column space of ; the theorem decomposes that projection by first removing the component spanned by . This is the algebraic content of "controlling for" a set of regressors: a coefficient is read as the effect of its regressor after netting out the linear influence of the controls. The fixed-effects within estimator is FWL applied with a full set of unit dummies, which is why demeaning delivers identical estimates.
Identification
A parameter is identified when the distribution of the observable data pins it down to a single value. In the linear model, MLR.1–MLR.3 identify because the moment condition has a unique solution whenever is nonsingular. When strict exogeneity fails, that moment condition no longer holds at the true , and OLS converges instead to the linear projection coefficient — the coefficient of the best linear predictor of given — which need not equal the causal effect of interest. Identification, not estimation, is where causal content enters: an estimator can be consistent for the projection coefficient yet irrelevant for the policy question.
Economic theory Intermediate+
Gauss-Markov theorem. Under MLR.1–MLR.4, the OLS estimator is the best linear unbiased estimator (BLUE) of : among all estimators of the form with a function of alone and , OLS has the smallest conditional variance matrix in the matrix sense,
Proof. Write , so . Let be any linear unbiased estimator; conditional unbiasedness forces . Decompose . Then . Now , and
because the cross term vanishes via . Since is positive semidefinite, .
Two cautions follow. BLUE is a ranking among linear unbiased estimators; once homoscedasticity (MLR.4) fails, OLS remains unbiased but the efficient estimator is the heteroscedasticity-robust GLS/Feasible-GLS, and robust "sandwich" standard errors replace the classical ones. And BLUE says nothing about consistency or causality: a biased-but-consistent IV estimator can dominate OLS when MLR.3 fails, even though it is not unbiased in finite samples.
Bridge. This result builds toward 52.04.01 (game theory and structural econometrics), where endogenous strategic regressors break MLR.3 and force instrumental-variables or structural estimation, and appears again in 45.07.01 (statistical learning theory), whose bias-variance trade-off generalises the BLUE ranking to biased estimators such as ridge regression. The foundational reason Gauss-Markov matters is that it fixes OLS as the reference point against which every alternative estimator is judged, and the central insight — projecting onto the column space of — is exactly the geometric fact that makes OLS a conditional expectation under MLR.1–MLR.3; putting these together, the bridge is that the same orthogonal-projection geometry recurs from the classical linear model to GMM and to high-dimensional inference.
Exercises Intermediate+
Lean formalization Intermediate+
lean_status: none. Econometrics is a statistical and research-design discipline rather than a theorem-proving one: its correctness gate is the validity of identifying assumptions, the consistency of estimators under those assumptions, and the quality of the data. The underlying mathematics — laws of large numbers, central-limit theorems, and the matrix algebra of orthogonal projection — does belong to probability [37.x] and mathematical statistics [45.x], and the corresponding Mathlib gap analysis documents which pieces of an OLS / IV / GMM formalisation Mathlib does not yet provide.
Advanced results Master
Instrumental variables and 2SLS. When MLR.3 fails but a vector of instruments is available with (exclusion/exogeneity) and (relevance/rank), is again identified. With instruments, the two-stage least squares estimator is
the projection of onto the column space of . Under iid sampling, is consistent and asymptotically normal with a heteroscedasticity-robust sandwich variance. When (overidentification), Hansen's -statistic, under the null, tests whether the overidentifying restrictions are jointly valid.
Generalised method of moments (GMM). Hansen (1982) subsumes IV. Given moment conditions with and sample average , the GMM estimator minimises in a positive-definite weighting matrix . The optimal weight , the inverse of the long-run variance of the moments, yields the efficient two-step GMM estimator, and is the overidentifying-restrictions test for the whole GMM class. Two-stage least squares is the GMM special case under homoscedasticity [Wooldridge Cross Section].
Panel data. With units observed over periods , the model carries a unit-specific effect . Fixed-effects (within) estimation demeans by unit, purging the time-invariant that would otherwise confound the estimate; the price is the loss of estimates on time-invariant regressors. Random-effects GLS treats as random and uncorrelated with , gaining efficiency if the orthogonality holds. The Hausman test compares the two: a statistically significant gap indicates that is correlated with the regressors and random effects is inconsistent, so fixed effects is preferred.
Time series. For time-ordered data, stationarity — a constant mean, variance, and autocovariance structure over time — is the regularity condition under which standard asymptotic theory applies. Regressing one independent random walk on another produces a spurious regression (Granger and Newbold, 1974): conventional -statistics diverge even though no genuine relationship exists. Cointegration (Engle and Granger, 1987) restores inference when non-stationary series share a stochastic trend: a linear combination of them is stationary, and an error-correction model estimates both the long-run equilibrium relationship and the short-run speed of adjustment back toward it [Hamilton Time Series].
Potential outcomes and the LATE theorem. In the treatment-effects notation each unit has potential outcomes and and a binary treatment ; the observed outcome is . Let be a binary instrument. Under independence of from the potential outcomes and potential treatments, exclusion ( affects only through ), relevance, and monotonicity (no unit takes the treatment when the instrument discourages it), the IV estimand identifies the local average treatment effect — the average effect of treatment on the compliers whose treatment status responds to the instrument [Angrist Imbens 1994]. This converts IV from a structural-coefficient estimator into a design-based causal estimand with an unambiguous population-level meaning, at the cost of estimating only a local effect that may not transport to non-compliers.
A contested identification question. Two research programmes answer "how do we learn causal effects from observational data?" in genuinely different ways. The design-based / natural-experiment school (Angrist, Card, Imbens; "Mostly Harmless Econometrics") hunts for transparent sources of exogenous variation — Vietnam-era draft lotteries, policy discontinuities at borders, weather shocks — and reads the instrumental-variables estimand as the LATE for the subpopulation whose treatment status the instrument shifts. This school prizes internal validity and the credibility of its identifying assumptions.
The structural school builds an explicit economic model of agent optimisation and market equilibrium, estimates its deep parameters, and then runs counterfactual policy experiments within the estimated model. Its defenders prize external validity — the ability to answer questions no single experiment can address, such as the effect of a tax regime that has never been observed. The mutual criticism is sharp: design estimates can be local and non-generalisable, while structural estimates inherit every assumption of the underlying model. Modern applied work increasingly blends the two, using natural experiments to identify a key elasticity and a structural model to extrapolate from it.
Synthesis. The econometric framework builds toward 52.04.01 (game-theoretic and structural econometrics), where strategic endogeneity demands explicit instruments and structural models, and appears again in 45.07.01 (statistical learning theory), whose bias-variance analysis generalised the BLUE ranking to penalised estimators. The foundational reason the Gauss-Markov theorem and the identification analysis matter together is that they separate what data alone can deliver from what requires assumptions: OLS is a description of conditional correlation, and causal content enters only through exogeneity or an instrument; this is exactly the distinction that fixed-effects, IV, and GMM each operationalise. Putting these together, the central insight is that credible causal inference from observational data is a research-design problem — what variation identifies what parameter — the bridge is that every estimator (OLS, 2SLS, within, GMM) is a moment condition evaluated under identifying assumptions, and the pattern generalises to the potential-outcomes and difference-in-differences frameworks that dominate modern applied microeconomics.
Full proof set Master
Proposition (Asymptotic normality of OLS). Under MLR.1–MLR.4 with iid observations and positive definite,
Proof. From and ,
where and .
Since are iid with finite expectation , the weak law of large numbers gives , and the continuous-mapping theorem gives (positive definiteness of keeps the inverse well-defined in probability limit). For the second factor, MLR.3 gives , and MLR.4 plus iterated expectations yields . By the Lindeberg–Lévy central-limit theorem, .
Combining, . Slutsky's theorem (, with the Gaussian limit independent of in the limit) gives the product limit , which equals .
Proposition (Consistency of 2SLS). Under valid instruments and the rank condition , .
Proof. Substituting ,
Scale the denominator and numerator by . The denominator converges in probability to , where and ; this matrix is nonsingular by the rank condition. The numerator converges to by the law of large numbers and instrument exogeneity. Hence .
Connections Master
Microeconomics
52.01.01. Econometrics supplies the empirical content for the microeconomic theory of demand, production, and market equilibrium: the supply and demand elasticities that theory treats as parameters are the objects OLS and IV are deployed to estimate, and the structural predictions of consumer and firm optimisation motivate the identifying assumptions.Linear model and Gauss-Markov BLUE
45.06.01. The statistical foundations of OLS — the linear model, the projection geometry, the BLUE theorem, and exact finite-sample inference under Gaussian errors — are developed in full measure-theoretic detail in the mathematical-statistics chapter; this unit is their economic application.Asymptotic statistics: M- and Z-estimators
45.04.04. GMM is the econometric realisation of the general Z-estimator programme: a vector of population moment conditions and its sample analogue define the estimator, and consistency with asymptotic normality follow from the uniform law of large numbers and the central-limit theorem treated there.Probability foundations: laws of large numbers and CLT
37.02.02,37.03.02. Every consistency and asymptotic-normality claim in this unit rests on the strong and weak laws of large numbers and on central-limit theorems (Lindeberg–Lévy, Lindeberg–Feller); the econometrician treats these as the engine that turns a finite sample into a trustworthy parameter estimate.
Historical & philosophical context Master
The probability foundations of econometrics were laid by Trygve Haavelmo in The Probability Approach in Econometrics (1944), which argued that economic laws should be stated as probability statements about joint distributions of observable variables, and that estimation and inference must therefore be conducted with the tools of mathematical statistics [Haavelmo 1944]. Haavelmo's move recast economics as an inferential science and made the distinction between correlation and causal structure a formal rather than a rhetorical matter. The Cowles Commission programme of the 1940s and 1950s — Koopmans, Marschak, Anderson, Rubin — built simultaneous-equations estimation and identification theory on this base, giving economics its characteristic vocabulary of structural parameters, reduced forms, and identification conditions.
The least-squares estimator itself is far older: Carl Friedrich Gauss introduced the method of least squares in astronomical work around 1809, deriving it as the maximum-likelihood estimator under Gaussian errors and noting its optimality properties. Legendre published the method independently in 1805. For most of the nineteenth and early twentieth centuries, least squares was a numerical curve-fitting tool; the Cowles synthesis showed how to embed it inside a probabilistic model of economic behaviour.
The late twentieth century brought a second revolution. Angrist and Imbens (1994) proved the local average treatment effect theorem, showing that under monotonicity an instrumental variable identifies the average treatment effect for the subgroup of compliers — a precise and modest causal estimand that recast IV as a tool of design-based causal inference rather than structural estimation [Angrist Imbens 1994]. This converted IV from a coefficient-estimator in a structural equation into a causal estimand tied to a well-defined subpopulation, and it supplied the formal scaffolding for the design-based programme.
The design-based turn — sometimes called the credibility revolution — reshaped labour, public, and development economics from the 1990s onward. Card and Krueger's 1994 minimum-wage study using a natural experiment across adjacent fast-food restaurants, and the diffusion of difference-in-differences and regression-discontinuity designs, made research design rather than estimation technique the locus of methodological debate. Parallel work on cluster-robust inference, the bootstrap, and weak instruments (Bound, Jaeger, and Baker, 1995; Staiger and Stock, 1997) supplied the inferential machinery that credible identification required. The structural tradition continued in industrial organisation and macroeconomics, and the contemporary discipline holds both programmes in productive tension: identification is the shared frontier, and the philosophical question — what variation in the world licenses a causal claim? — remains the discipline's defining one.
Bibliography Master
@book{StockWatson2020,
author = {Stock, James H. and Watson, Mark W.},
title = {Introduction to Econometrics},
edition = {4},
publisher = {Pearson},
year = {2020},
}
@book{WooldridgeIntro2020,
author = {Wooldridge, Jeffrey M.},
title = {Introductory Econometrics: A Modern Approach},
edition = {7},
publisher = {Cengage Learning},
year = {2020},
}
@book{WooldridgeCrossPanel2010,
author = {Wooldridge, Jeffrey M.},
title = {Econometric Analysis of Cross Section and Panel Data},
edition = {2},
publisher = {MIT Press},
year = {2010},
}
@book{AngristPischke2009,
author = {Angrist, Joshua D. and Pischke, Jörn-Steffen},
title = {Mostly Harmless Econometrics: An Empiricist's Companion},
publisher = {Princeton University Press},
year = {2009},
}
@book{Hamilton1994,
author = {Hamilton, James D.},
title = {Time Series Analysis},
publisher = {Princeton University Press},
year = {1994},
}
@article{Hansen1982,
author = {Hansen, Lars Peter},
title = {Large Sample Properties of Generalized Method of Moments Estimators},
journal = {Econometrica},
volume = {50},
number = {4},
pages = {1029--1054},
year = {1982},
}
@phdthesis{Haavelmo1944,
author = {Haavelmo, Trygve},
title = {The Probability Approach in Econometrics},
school = {Harvard University},
year = {1944},
}
@article{AngristImbens1994,
author = {Angrist, Joshua D. and Imbens, Guido W.},
title = {Identification and Estimation of Local Average Treatment Effects},
journal = {Econometrica},
volume = {62},
number = {2},
pages = {467--475},
year = {1994},
}
@article{EngleGranger1987,
author = {Engle, Robert F. and Granger, Clive W. J.},
title = {Co-Integration and Error Correction: Representation, Estimation, and Testing},
journal = {Econometrica},
volume = {55},
number = {2},
pages = {251--276},
year = {1987},
}