The Bias-Variance Decomposition and the Prediction-Error Trade-off

Intuition Beginner

When you fit a model to data and then use it to predict something new, your prediction can be wrong for three different reasons, and it helps to keep them apart. The first reason is plain bad luck: the thing you are predicting has randomness in it that no model could ever pin down. The second reason is that your model is too simple to capture the real pattern, so it is systematically off. The third reason is that your model is so flexible that it chases the random wiggles in your particular dataset, so it would have come out quite differently with a different sample.

These three reasons have names. The irreducible noise is the bad luck you can never remove. The bias is how far your model is, on average, from the truth because it is too rigid. The variance is how much your fitted model jumps around as the training data changes. The bias-variance trade-off is the observation that the second and third usually pull against each other: making a model more flexible cuts its bias but raises its variance, and making it more rigid does the reverse.

Think of throwing darts at a board. A beginner who always pulls left is biased but consistent: the darts cluster, just in the wrong place. A jittery expert aims at the center but scatters widely: low bias, high variance. The best dart thrower wants both the cluster and the center. A prediction model faces the same tension, and the total error you should expect is the noise plus the squared bias plus the variance.

This is why "more complex model" is not the same as "better model." As you add flexibility, error first drops because bias falls, then climbs again because variance grows. The sweet spot in the middle is what model selection and regularization are trying to find.

Visual Beginner

The classic picture is a U-shaped curve. As model complexity increases from left to right, the error you would make on fresh data first falls, bottoms out, and then rises. Underneath it, the bias keeps falling and the variance keeps rising, and their sum traces the U.

Model complexity	Bias	Variance	Test error (noise + bias² + variance)
Very low (too rigid)	High	Low	High (dominated by bias)
Moderate (sweet spot)	Medium	Medium	Lowest
Very high (too flexible)	Low	High	High (dominated by variance)

The training error (how well you fit the data you learned from) keeps dropping as complexity rises, even past the sweet spot. That gap between the falling training curve and the rising test curve is the visible signature of overfitting.

Worked example Beginner

Suppose the true relationship is that the average response at input $x=1$ equals $10$, and any single measurement adds noise with a spread of $2$ (so the noise variance is $4$). We compare two methods by imagining many repeated datasets and seeing what each predicts at $x=1$.

Method A is a rigid flat line. Across many datasets its prediction at $x=1$ is always close to $8$, almost never moving. So its bias is $10-8=2$, its squared bias is $4$, and its variance is about $0$ because it barely changes.

Method B is a wiggly curve. Across many datasets its predictions at $x=1$ average out to $10$ (no bias), but they swing between $6$ and $14$, giving a variance of about $9$.

Now add up the expected squared prediction error, which is noise plus squared bias plus variance.

For Method A: $4+4+0=8$.

For Method B: $4+0+9=13$.

Method A wins here, $8$ versus $13$, even though Method B is unbiased. The wiggly method pays for its flexibility with variance. What this tells us: being right on average is not enough. A method that is a little biased but stable can beat an unbiased method that is all over the place, and the decomposition is what lets you see the comparison cleanly.

Check your understanding Beginner

Formal definition Intermediate+

Fix a regression setting. The response $Y$ and predictor $X$ are random with joint law, and we model $Y=f(X)+\epsilon$ where $f(x)=\mathbb{E}[Y\mid X=x]$ is the regression function [from 26.06.01] and the noise satisfies $\mathbb{E}[\epsilon \mid X]=0$ and $\mathrm{Var}(\epsilon \mid X=x)={\sigma }^2$. The expectation and variance operators are those of 26.03.01.

A training sample $\mathcal{D}=\{(X_1,Y_1),\dots ,(X_n,Y_n)\}$ is drawn i.i.d. from this law. A learning procedure produces a fitted predictor ${\hat{f}}_{\mathcal{D}}:\mathcal{X}\to \mathbb{R}$, a measurable function of $\mathcal{D}$. We evaluate it at a fixed query point $x_0$, treating ${\hat{f}}_{\mathcal{D}}(x_0)$ as a random variable whose randomness comes from $\mathcal{D}$.

Definition (pointwise prediction error). The expected squared prediction error at $x_0$ for a fresh response $Y_0=f(x_0)+{\epsilon }_0$ independent of $\mathcal{D}$ is

$$\mathrm{Err}(x_0)={\mathbb{E}}_{\mathcal{D},{\epsilon }_0}\left[(Y_0-{\hat{f}}_{\mathcal{D}}(x_0){)}^2\right].$$

Definition (bias and variance of the procedure). Write $\bar{f}(x_0)={\mathbb{E}}_{\mathcal{D}}[{\hat{f}}_{\mathcal{D}}(x_0)]$ for the average fit over training samples. The bias and variance of the procedure at $x_0$ are

$$\mathrm{Bias}(x_0)=\bar{f}(x_0)-f(x_0),\mathrm{Var}(x_0)={\mathbb{E}}_{\mathcal{D}}\left[({\hat{f}}_{\mathcal{D}}(x_0)-\bar{f}(x_0){)}^2\right].$$

The integrated (population-averaged) versions replace $x_0$ by a random query $X_0$ drawn from the marginal of $X$: $\overline{\mathrm{Err}}={\mathbb{E}}_{X_0}[\mathrm{Err}(X_0)]$, and likewise for integrated squared bias and integrated variance.

Definition (in-sample, out-of-sample, training error). With fitted values ${\hat{y}}_i={\hat{f}}_{\mathcal{D}}(X_i)$, the training error is $\overline{\mathrm{err}}=\frac{1}{n}{\sum }_{i=1}^n(Y_i-{\hat{y}}_i{)}^2$. The in-sample error re-draws fresh responses $Y_i^{\text{new}}$ at the same training inputs $X_i$ and averages $\frac{1}{n}{\sum }_i\mathbb{E}[(Y_i^{\text{new}}-{\hat{y}}_i{)}^2]$. The out-of-sample (generalization) error is $\mathrm{Err}(X_0)$ at an independently drawn $X_0$. The optimism is the gap $\mathbb{E}[\text{in-sample error}]-\mathbb{E}[\overline{\mathrm{err}}]$, the amount by which training error understates true error.

Counterexamples to common slips Intermediate+

• Bias is a property of the procedure, not of one fitted model. $\mathrm{Bias}(x_0)$ involves the average $\bar{f}(x_0)$ over the sampling distribution of $\mathcal{D}$. A single fit being far from $f(x_0)$ is not bias; it could be variance.
• Unbiased does not mean low error. An interpolating fit can have zero bias and enormous variance, with worse $\mathrm{Err}$ than a biased but stable competitor. The worked example above is exactly this case.
• The noise floor is unavoidable only when error is measured against $Y_0$, not against $f(x_0)$. The ${\sigma }^2$ term appears because we predict a noisy response; the squared error against the deterministic $f(x_0)$ has no ${\sigma }^2$ term.
• Training error is not an estimate of generalization error. It is biased downward by the optimism, which is positive whenever the fitted values are positively correlated with the responses they were fit to.

Key theorem with proof Intermediate+

Theorem (pointwise bias-variance decomposition for squared-error loss). Under the regression model $Y_0=f(x_0)+{\epsilon }_0$ with $\mathbb{E}[{\epsilon }_0]=0$, $\mathrm{Var}({\epsilon }_0)={\sigma }^2$, and ${\epsilon }_0$ independent of the training sample $\mathcal{D}$,

$$\mathrm{Err}(x_0)=\underset{\text{irreducible}}{\underbrace{{\sigma }^2}}+\underset{\mathrm{Bias}(x_0{)}^2}{\underbrace{(\bar{f}(x_0)-f(x_0){)}^2}}+\underset{\mathrm{Var}(x_0)}{\underbrace{{\mathbb{E}}_{\mathcal{D}}[({\hat{f}}_{\mathcal{D}}(x_0)-\bar{f}(x_0){)}^2]}}.$$

Integrating over $X_0\sim P_X$ gives $\overline{\mathrm{Err}}={\sigma }^2+{\mathbb{E}}_{X_0}[\mathrm{Bias}(X_0{)}^2]+{\mathbb{E}}_{X_0}[\mathrm{Var}(X_0)]$.

Proof. Abbreviate $\hat{f}={\hat{f}}_{\mathcal{D}}(x_0)$, $\bar{f}=\bar{f}(x_0)={\mathbb{E}}_{\mathcal{D}}[\hat{f}]$, $f=f(x_0)$. Insert and subtract $f$ inside the square, using $Y_0=f+{\epsilon }_0$:

$$(Y_0-\hat{f}{)}^2=({\epsilon }_0+f-\hat{f}{)}^2={\epsilon }_0^2+2{\epsilon }_0(f-\hat{f})+(f-\hat{f}{)}^2.$$

Take ${\mathbb{E}}_{\mathcal{D},{\epsilon }_0}$. Since ${\epsilon }_0$ is independent of $\mathcal{D}$ with mean zero, $\mathbb{E}[{\epsilon }_0(f-\hat{f})]=\mathbb{E}[{\epsilon }_0]\mathbb{E}[f-\hat{f}]=0$, and $\mathbb{E}[{\epsilon }_0^2]={\sigma }^2$. The cross term vanishes, leaving

$$\mathrm{Err}(x_0)={\sigma }^2+{\mathbb{E}}_{\mathcal{D}}[(f-\hat{f}{)}^2].$$

Now decompose the remaining term by inserting and subtracting $\bar{f}$:

$$(f-\hat{f}{)}^2=((f-\bar{f})+(\bar{f}-\hat{f}){)}^2=(f-\bar{f}{)}^2+2(f-\bar{f})(\bar{f}-\hat{f})+(\bar{f}-\hat{f}{)}^2.$$

The middle term has expectation zero over $\mathcal{D}$ because $f-\bar{f}$ is a constant and ${\mathbb{E}}_{\mathcal{D}}[\bar{f}-\hat{f}]=\bar{f}-{\mathbb{E}}_{\mathcal{D}}[\hat{f}]=0$. Therefore

$${\mathbb{E}}_{\mathcal{D}}[(f-\hat{f}{)}^2]=(f-\bar{f}{)}^2+{\mathbb{E}}_{\mathcal{D}}[(\hat{f}-\bar{f}{)}^2]=\mathrm{Bias}(x_0{)}^2+\mathrm{Var}(x_0).$$

Combining gives the pointwise identity. The integrated form follows by applying ${\mathbb{E}}_{X_0}$ to both sides and using linearity, with ${\sigma }^2$ assumed constant in $x_0$ (otherwise it is replaced by ${\mathbb{E}}_{X_0}[{\sigma }^2(X_0)]$). $\square$

Bridge. This decomposition builds toward every regularization and model-selection method in this chapter, because each of them is a deliberate move along the bias-variance axis: shrinkage trades a controlled increase in bias for a larger decrease in variance, and the optimal tuning is the point where the two marginal changes balance. The foundational reason the identity is exactly an orthogonal split is that squared-error loss is a squared $L^2$ norm and the three pieces are mutually orthogonal deviations — noise around $f$, the average fit's deviation from $f$, and the fit's deviation from its own average — so this is exactly the Pythagorean theorem in the Hilbert space of square-integrable random variables. The same three-term split appears again in the optimism-of-training-error identity, where the variance term re-emerges as a covariance between fitted values and responses; and it generalises beyond the constant-noise case by replacing ${\sigma }^2$ with its conditional value. Putting these together, the decomposition is the quantitative skeleton on which ridge regression, subset selection, and cross-validation all hang.

Exercises Intermediate+

Advanced results Master

The decomposition is an instance of a single Hilbert-space identity. In $L^2(\Omega )$ with inner product $⟨U,V⟩=\mathbb{E}[UV]$, the random response $Y_0$, the average fit $\bar{f}(x_0)$, and the random fit ${\hat{f}}_{\mathcal{D}}(x_0)$ generate three orthogonal deviations: ${\epsilon }_0=Y_0-f(x_0)$, the constant $f(x_0)-\bar{f}(x_0)$, and $\bar{f}(x_0)-{\hat{f}}_{\mathcal{D}}(x_0)$. The first is mean-zero and independent of $\mathcal{D}$; the third is mean-zero over $\mathcal{D}$; the middle is deterministic. Pairwise inner products vanish, and $\mathrm{Err}(x_0)=\Vert {\epsilon }_0{\Vert }^2+\Vert f-\bar{f}{\Vert }^2+\Vert \bar{f}-\hat{f}{\Vert }^2$ is the Pythagorean sum. Every refinement of the trade-off is a refinement of this orthogonality.

Effective degrees of freedom make the variance term operational. For a linear smoother $\hat{\mathbf{y}}=\mathbf{Sy}$, the quantity $\mathrm{df}(\mathbf{S})=\mathrm{tr}(\mathbf{S})$ measures model complexity continuously, interpolating between the discrete parameter count of subset selection and the smooth shrinkage of ridge. The general definition $\mathrm{df}=\frac{1}{{\sigma }^2}{\sum }_i\mathrm{Cov}({\hat{y}}_i,Y_i)$ ties degrees of freedom directly to optimism through the identity of the previous section, so that $\mathbb{E}[\text{in-sample error}]=\mathbb{E}[\overline{\mathrm{err}}]+\frac{2{\sigma }^2}{n}\mathrm{df}$. Mallows' $C_p$, Akaike's information criterion, and the generalized cross-validation score are all estimators of in-sample error built by plugging an estimate of $\mathrm{df}$ into this relation, and each is a computable surrogate for the U-shaped test curve.

For ridge regression with design $\mathbf{X}$ and penalty $\lambda$, the fit is ${\hat{\mathbf{\beta }}}_{\lambda }=({\mathbf{X}}^{\top }\mathbf{X}+\lambda \mathbf{I}{)}^{-1}{\mathbf{X}}^{\top }\mathbf{y}$ with smoother ${\mathbf{S}}_{\lambda }=\mathbf{X}({\mathbf{X}}^{\top }\mathbf{X}+\lambda \mathbf{I}{)}^{-1}{\mathbf{X}}^{\top }$ and $\mathrm{df}(\lambda )={\sum }_j\frac{d_j^2}{d_j^2+\lambda }$, where $d_j$ are the singular values of $\mathbf{X}$. As $\lambda$ runs from $0$ to $\infty$, $\mathrm{df}$ runs from $\mathrm{rank}(\mathbf{X})$ down to $0$, the squared bias rises monotonically, and the variance ${\sigma }^2{\sum }_j\frac{d_j^2}{(d_j^2+\lambda {)}^2}$ falls monotonically. There is always a positive $\lambda$ strictly improving on least squares in integrated MSE, the multivariate generalization of Exercise 5, which is the precise content of the claim that some shrinkage always helps.

The double-descent phenomenon shows that the classical U is the low-complexity half of a richer curve. As model complexity is pushed past the interpolation threshold — the point where the model has just enough capacity to fit the training data exactly — the test error, after the classical rise, can fall a second time, sometimes below the classical minimum. Belkin, Hsu, Ma and Mandal (2019) named and mapped this; in the overparametrized regime the variance is controlled not by parameter count but by the implicit regularization of the fitting algorithm, which among the many interpolating solutions selects the one of smallest norm. The bias-variance language survives intact: the second descent is a variance decrease driven by minimum-norm interpolation, and the classical analysis applies tier by tier to the effective, not the nominal, complexity.

Synthesis. The foundational reason this single identity organizes regularization, model selection, and even modern overparametrized learning is that squared-error loss is a squared norm, so prediction error is a sum of squared orthogonal deviations, and every method in the chapter is a controlled motion of those deviations. This is exactly the Pythagorean theorem in $L^2$, and putting these together with the optimism identity shows that the variance term and the training-error optimism are the same object viewed from two sides — a covariance between fit and data — which is why effective degrees of freedom simultaneously index complexity, govern overfitting, and define the penalties in $C_p$ and AIC. The central insight is that shrinkage is never free yet often profitable: the multivariate ridge calculation generalises the one-coordinate computation to show a strictly positive optimal penalty whenever the noise is nonzero, and this is dual to the learning-theoretic statement that estimation error must be paid down against approximation error. The double-descent picture does not overturn this; it shows the same decomposition operating on effective rather than nominal complexity, so the U-shaped curve is one regime of a structure that the orthogonal split describes everywhere.

Full proof set Master

The pointwise and integrated decompositions and the optimism identity are proved in full in the Key theorem section and Exercise 6. The remaining Master claims are recorded here.

Proposition (monotonicity of ridge bias and variance in $\lambda$). In the singular-value coordinates of $\mathbf{X}$, the ridge integrated squared bias is non-decreasing and the integrated variance is non-increasing in $\lambda \ge 0$, and there exists $\lambda >0$ with integrated MSE strictly below that of least squares whenever ${\sigma }^2>0$ and $\mathbf{X}$ is not degenerate.

Proof. Rotate to the basis where ${\mathbf{X}}^{\top }\mathbf{X}=\mathrm{diag}(d_1^2,\dots ,d_p^2)$. Coordinate $j$ of the ridge fit shrinks the least-squares coordinate by the factor $s_j(\lambda )=\frac{d_j^2}{d_j^2+\lambda }$, decreasing in $\lambda$ from $1$ to $0$. The squared bias in coordinate $j$ is $(1-s_j(\lambda ){)}^2{\beta }_j^2=(\frac{\lambda }{d_j^2+\lambda }{)}^2{\beta }_j^2$, increasing in $\lambda$; summing preserves monotonicity. The variance in coordinate $j$ is $s_j(\lambda {)}^2{\sigma }^2/d_j^2=\frac{d_j^2{\sigma }^2}{(d_j^2+\lambda {)}^2}$, decreasing in $\lambda$; summing preserves monotonicity. For the strict improvement, the derivative of the total MSE at $\lambda =0$ is ${\sum }_j\frac{d}{d\lambda }[(1-s_j{)}^2{\beta }_j^2+s_j^2{\sigma }^2/d_j^2]{|}_{\lambda =0}$. The bias derivative vanishes at $\lambda =0$ (it enters quadratically through ${\lambda }^2$), while the variance derivative is ${\sum }_j\frac{-2{\sigma }^2}{d_j^2}\cdot \frac{1}{d_j^2}\cdot d_j^2=-{\sum }_j\frac{2{\sigma }^2}{d_j^2}<0$. So MSE is strictly decreasing at $\lambda =0$, hence some $\lambda >0$ does strictly better. $\square$

Proposition (degrees of freedom equal optimism over $2{\sigma }^2/n$). For any procedure with fitted values at fixed inputs and homoscedastic noise, $\mathbb{E}[\text{in-sample error}]-\mathbb{E}[\overline{\mathrm{err}}]=\frac{2{\sigma }^2}{n}\mathrm{df}$ with $\mathrm{df}=\frac{1}{{\sigma }^2}{\sum }_i\mathrm{Cov}({\hat{y}}_i,Y_i)$.

Proof. Exercise 6 establishes optimism $=\frac{2}{n}{\sum }_i\mathrm{Cov}({\hat{y}}_i,Y_i)$. Substitute the definition ${\sum }_i\mathrm{Cov}({\hat{y}}_i,Y_i)={\sigma }^2\mathrm{df}$, giving optimism $=\frac{2{\sigma }^2}{n}\mathrm{df}$ directly. For a linear smoother $\hat{\mathbf{y}}=\mathbf{Sy}$ this specializes via Exercise 4 to $\mathrm{df}=\mathrm{tr}(\mathbf{S})$, and for least squares with the hat matrix $\mathrm{tr}(\mathbf{H})=p$ recovers the parameter count. $\square$

Proposition (0-1 loss has no universally additive decomposition). There is a binary classification problem and a randomized classifier ensemble for which increasing the variance of the predicted label strictly decreases the expected 0-1 loss; hence no decomposition with an always-non-negative, always-harmful variance term can hold for 0-1 loss.

Proof. Fix a point $x_0$ where the Bayes-optimal label is $1$ and the noise is ${\sigma }_{01}<1/2$, so the population responds with label $1$ with probability $1-{\sigma }_{01}>1/2$. Consider a classifier whose predicted label at $x_0$ is $1$ with probability $q$ over the draw of $\mathcal{D}$. The expected 0-1 loss is $q{\sigma }_{01}+(1-q)(1-{\sigma }_{01})$, decreasing in $q$. A higher-variance ensemble that nonetheless raises $q$ toward $1$ — for instance by majority voting that concentrates predictions on the more probable label — lowers the loss. Thus the effect of prediction variability on 0-1 loss can be strictly beneficial, which no decomposition with a fixed-sign variance term can represent. The squared-error orthogonality that forces a non-negative variance term has no 0-1 analogue. $\square$

Connections Master

Correlation and regression analysis 26.06.01 supplies the squared-error, least-squares prediction framework this unit decomposes; the regression function $f(x)=\mathbb{E}[Y\mid X=x]$ defined there is the target against which bias is measured, and the hat-matrix optimism here is the inferential counterpart of the residual analysis introduced for ordinary least squares.

Random variables and expected value 26.03.01 provides the expectation and variance algebra on which the entire decomposition runs; the Pythagorean split is an identity in the $L^2$ inner product $⟨U,V⟩=\mathbb{E}[UV]$, and the orthogonality of mean-zero deviations used in the proof is exactly the uncorrelatedness machinery of that unit.

Characteristic functions and the Lévy continuity theorem 37.03.01 sit one level up: the moment functionals recovered there by differentiating the transform at the origin are the same first and second moments that define bias and variance, and the central-limit behaviour governs how the variance term of an averaging estimator such as $k$-nearest-neighbours shrinks as the sample grows.

Sampling distributions and the central limit theorem 26.04.01 formalize the over-$\mathcal{D}$ randomness that bias and variance are expectations against; the sampling distribution of ${\hat{f}}_{\mathcal{D}}(x_0)$ is precisely the object whose mean gives $\bar{f}$ and whose spread gives the variance term.

Bayesian statistics 26.07.01 reframes shrinkage as a posterior mean under a prior: ridge regression is the maximum-a-posteriori estimate under a Gaussian prior on the coefficients, so the bias deliberately introduced here is the prior pull, and the optimal penalty of Exercise 5 is the prior-to-noise variance ratio.

Nonparametric methods 26.08.01 are where the trade-off is most visible: the bandwidth or smoothing parameter directly indexes effective degrees of freedom, and selecting it by cross-validation is selecting the minimum of the U-shaped curve this unit anatomizes. The forthcoming ridge-regression and model-selection units in this chapter, and the learning-theoretic approximation-estimation trade-off of the statistical-learning-theory chapter, are the same balance expressed in shrinkage and in capacity-control language respectively.

Historical & philosophical context Master

The mean-squared-error split into squared bias and variance is old, implicit in Gauss's least-squares analysis and explicit in early-twentieth-century estimation theory, but its elevation to an organizing principle of learning is due to Stuart Geman, Elie Bienenstock and René Doursat, whose 1992 paper Neural networks and the bias/variance dilemma ^{[Geman 1992]} argued that the practical difficulty of nonparametric and neural-network learning is precisely the management of variance, and that consistency results guaranteeing zero asymptotic bias say little about finite-sample behaviour where variance dominates. The full prediction-error treatment, including the in-sample/out-of-sample distinction and the optimism correction through effective degrees of freedom, was consolidated by Hastie, Tibshirani and Friedman ^{[Hastie 2009]} in chapters 2 and 7 of The Elements of Statistical Learning, which made the U-shaped test-error curve the standard mental model for model selection.

Pedro Domingos ^{[Domingos 2000]} showed that the decomposition is loss-family-specific: squared-error loss is the case with a strictly additive, always-harmful variance term, while 0-1 loss admits regions where variance reduces error, explaining the empirical success of high-variance, low-bias classifiers combined by voting. The classical picture was complicated again by Mikhail Belkin, Daniel Hsu, Siyuan Ma and Soumik Mandal ^{[Belkin 2019]}, whose double-descent curve demonstrated that test error can descend a second time beyond the interpolation threshold, reconciling the textbook trade-off with the observed generalization of heavily overparametrized models through the implicit norm-minimizing regularization of the fitting algorithm.

Bibliography Master

@article{geman1992,
  author  = {Geman, Stuart and Bienenstock, Elie and Doursat, Ren\'e},
  title   = {Neural Networks and the Bias/Variance Dilemma},
  journal = {Neural Computation},
  volume  = {4},
  number  = {1},
  pages   = {1--58},
  year    = {1992}
}

@book{hastie2009,
  author    = {Hastie, Trevor and Tibshirani, Robert and Friedman, Jerome},
  title     = {The Elements of Statistical Learning: Data Mining, Inference, and Prediction},
  edition   = {2nd},
  publisher = {Springer-Verlag, New York},
  year      = {2009}
}

@inproceedings{domingos2000,
  author    = {Domingos, Pedro},
  title     = {A Unified Bias-Variance Decomposition and Its Applications},
  booktitle = {Proceedings of the 17th International Conference on Machine Learning},
  pages     = {231--238},
  year      = {2000}
}

@article{belkin2019,
  author  = {Belkin, Mikhail and Hsu, Daniel and Ma, Siyuan and Mandal, Soumik},
  title   = {Reconciling Modern Machine-Learning Practice and the Classical Bias-Variance Trade-off},
  journal = {Proceedings of the National Academy of Sciences},
  volume  = {116},
  number  = {32},
  pages   = {15849--15854},
  year    = {2019}
}

@book{james2013,
  author    = {James, Gareth and Witten, Daniela and Hastie, Trevor and Tibshirani, Robert},
  title     = {An Introduction to Statistical Learning},
  publisher = {Springer-Verlag, New York},
  year      = {2013}
}