Simultaneous diagonalisation of two quadratic forms and the generalised eigenvalue problem

Intuition Beginner

A single symmetric matrix has a set of perpendicular axes along which it acts by pure stretching. The spectral theorem 01.01.13 finds them. But many problems in physics and statistics come with two such matrices at once, and the two sets of axes usually point in different directions. The question of this unit is whether one clever choice of coordinates can straighten out both matrices together.

The surprising answer is yes, as long as one of the two matrices measures a genuine size, meaning it is positive whenever you feed it a nonzero vector. That positive matrix gets used as a new ruler. Once you agree to measure lengths and angles with that ruler instead of the ordinary one, the second matrix turns out to have perpendicular stretch axes in the new sense, and those very axes diagonalise both matrices at the same time.

A vibrating system makes this concrete. Two masses joined by springs have a kinetic-energy matrix and a potential-energy matrix. Diagonalising both at once finds the special motions, called normal modes, in which the whole system oscillates at a single clean frequency, with every part moving in step.

Visual Beginner

Two quadratic forms in the plane draw two ellipses centred at the origin. The first form, the positive-definite one, draws a reference ellipse; the second draws another ellipse, tilted differently. Ordinary perpendicular axes line up with at most one of them. The simultaneous-diagonalisation directions are the axes that look like the principal axes of both ellipses once you stretch the picture so the reference ellipse becomes a perfect circle.

The right panel is the punchline: after rescaling space so the blue reference ellipse becomes a circle, the red ellipse lines up with the coordinate axes. Those axes are the directions that diagonalise both forms together.

Worked example Beginner

Take two symmetric matrices in the plane,

$$A=\left(\begin{array}{cc}3& 1\\ 1& 3\end{array}\right),B=\left(\begin{array}{cc}2& 0\\ 0& 1\end{array}\right).$$

Here $B$ is positive: feeding it any nonzero vector $(x,y)$ gives $2x^2+y^2>0$. The generalised eigenvalues solve $\det (A-\lambda B)=0$.

Step 1. Form $A-\lambda B=\left(\begin{array}{cc}3-2\lambda & 1\\ 1& 3-\lambda \end{array}\right)$.

Step 2. Set its determinant to zero: $(3-2\lambda )(3-\lambda )-1=0$.

Step 3. Multiply out: $9-3\lambda -6\lambda +2{\lambda }^2-1=2{\lambda }^2-9\lambda +8=0$.

Step 4. Solve the quadratic: $\lambda =(9\pm \sqrt{81-64})/4=(9\pm \sqrt{17})/4$. Numerically ${\lambda }_1\approx 3.28$ and ${\lambda }_2\approx 1.22$.

Step 5. Find the first generalised eigenvector from $Av={\lambda }_{1}Bv$, that is $(A-{\lambda }_{1}B)v=0$. The top row gives $(3-2{\lambda }_1)x+y=0$, so $y=(2{\lambda }_1-3)x\approx 3.56x$. A direction is $v_1\approx (1,3.56)$.

Step 6. Check $B$-orthogonality of the two eigenvectors. The second eigenvector is $v_2\approx (1,-0.56)$. Then $v_1^{\top }B{v}_2=2(1)(1)+(3.56)(-0.56)\approx 2-1.99\approx 0$, confirming the eigenvectors are perpendicular when measured by $B$.

What this tells us: the two generalised eigenvalues are the values of $\lambda$ where $A$ and $\lambda B$ agree as forms in some direction, and the two eigenvectors are perpendicular in the ruler set by $B$, not in the ordinary ruler. Those directions are exactly the ones that diagonalise both forms at once.

Check your understanding Beginner

Formal definition Intermediate+

Let $V$ be a finite-dimensional real vector space and let $A,B$ be real symmetric bilinear forms on $V$ in the sense of 01.01.15, with Gram matrices (also written $A,B$) in a fixed basis. Assume $B$ is positive-definite, written $B\succ 0$: $v^{\top }Bv>0$ for every $v\ne 0$. Symmetry means $A=A^{\top }$ and $B=B^{\top }$. All forms in this unit are real symmetric unless stated otherwise.

Because $B\succ 0$, the formula

$$⟨u,v{⟩}_B=u^{\top }Bv$$

defines a genuine inner product on $V$, the $B$-inner product, with norm $\Vert v{\Vert }_B=\sqrt{v^{\top }Bv}$. Two vectors are $B$-orthogonal when $⟨u,v{⟩}_B=0$; a basis is $B$-orthonormal when its Gram matrix against $⟨\cdot ,\cdot {⟩}_B$ is the identity. This makes $(V,⟨\cdot ,\cdot {⟩}_B)$ a finite-dimensional inner-product space in the sense of 01.01.09.

Definition (generalised eigenvalue problem). A scalar $\lambda \in \mathbb{R}$ and a vector $v\ne 0$ form a generalised eigenpair of the ordered pair $(A,B)$ when

$$Av=\lambda Bv.$$

The scalar $\lambda$ is a generalised eigenvalue and $v$ a generalised eigenvector. Equivalently $\lambda$ is a root of the generalised characteristic polynomial

$$p(\lambda )=\det (A-\lambda B),$$

and $v\in \ker (A-\lambda B)$. The formal object $A-\lambda B$, viewed as a one-parameter family of matrices, is the matrix pencil of the pair; it is regular when $p$ is not identically zero, which holds automatically here because $\det (A-\lambda B)$ has leading term $(-\lambda {)}^n\det B\ne 0$.

Definition (simultaneous diagonalisation by congruence). The pair $(A,B)$ is simultaneously diagonalisable by congruence when there is an invertible $P$ with

$$P^{\top }BP=I,P^{\top }AP=\Lambda =\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n).$$

The transformation $X\mapsto P^{\top }XP$ is congruence, the change-of-variables law for bilinear forms 01.01.17, and it is the correct equivalence here: a substitution $v=Py$ sends the form $v^{\top }Xv$ to $y^{\top }(P^{\top }XP)y$. Congruence is not similarity ($P^{-1}XP$); the two coincide only when $P$ is orthogonal, and the diagonalising $P$ below is generally not orthogonal.

Definition (generalised Rayleigh quotient). For $v\ne 0$,

$$\rho (v)=\frac{v^{\top }Av}{v^{\top }Bv}$$

is the generalised Rayleigh quotient of $(A,B)$. It is well defined because $B\succ 0$ keeps the denominator positive, and it reduces to the ordinary Rayleigh quotient of 01.01.14 when $B=I$.

Counterexamples to common slips

• Positive-definiteness of $B$, not of $A$, is what the construction needs. The matrix $A$ may be indefinite; its generalised eigenvalues are then a mix of positive and negative reals, and the simultaneous diagonalisation still exists.
• The diagonalising $P$ is generally not orthogonal, so $P^{\top }AP$ is congruent but not similar to $A$. Reading off "eigenvalues of $A$" from $\Lambda$ is the error: the ${\lambda }_k$ are eigenvalues of the pencil, equivalently of $B^{-1}A$, not of $A$.
• If $B$ is merely positive-semidefinite or indefinite, the pencil may be defective or singular. The pair $A=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$, $B=\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right)$ has $\det (A-\lambda B)={\lambda }^2+1$, with no real generalised eigenvalue; no real congruence diagonalises both.

Key theorem with proof Intermediate+

Theorem (simultaneous diagonalisation of two forms, one definite; Shilov Ch. 10 ^{[source pending]}; Horn-Johnson §4.5 ^{[source pending]}). Let $A,B$ be real symmetric forms on an $n$-dimensional real vector space with $B\succ 0$. Then there is an invertible $P$ with

$$P^{\top }BP=I,P^{\top }AP=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n),$$

the columns of $P$ are $B$-orthonormal generalised eigenvectors, and the diagonal entries ${\lambda }_1\ge \cdots \ge {\lambda }_n$ are the generalised eigenvalues, that is the roots of $\det (A-\lambda B)=0$. All ${\lambda }_k$ are real.

Proof. Equip $V$ with the $B$-inner product $⟨u,v{⟩}_B=u^{\top }Bv$, which is a genuine inner product because $B\succ 0$. Consider the linear operator $T=B^{-1}A$. It is self-adjoint for $⟨\cdot ,\cdot {⟩}_B$: for all $u,v$,

$$⟨Tu,v{⟩}_B=(B^{-1}Au{)}^{\top }Bv=u^{\top }{A}^{\top }{B}^{-\top }Bv=u^{\top }Av,$$

using $A^{\top }=A$ and $B^{-\top }B=I$, and symmetrically $⟨u,Tv{⟩}_B=u^{\top }B(B^{-1}Av)=u^{\top }Av$. The two are equal, so $T=T^{\ast }$ in the $B$-inner product.

By the spectral theorem 01.01.13 applied to the self-adjoint $T$ on the inner-product space $(V,⟨\cdot ,\cdot {⟩}_B)$, there is a $⟨\cdot ,\cdot {⟩}_B$-orthonormal basis $p_1,\dots ,p_n$ of eigenvectors, $T{p}_k={\lambda }_k{p}_k$ with ${\lambda }_k\in \mathbb{R}$. The eigenvalue equation $B^{-1}A{p}_k={\lambda }_k{p}_k$ rearranges to $A{p}_k={\lambda }_{k}B{p}_k$, so each $p_k$ is a generalised eigenvector of $(A,B)$ with generalised eigenvalue ${\lambda }_k$.

Assemble the eigenvectors as the columns of $P=[p_1\mid \cdots \mid p_n]$. The $B$-orthonormality $⟨p_i,p_j{⟩}_B=p_i^{\top }B{p}_j={\delta }_{ij}$ is exactly $P^{\top }BP=I$. For the other form,

$$(P^{\top }AP{)}_{ij}=p_i^{\top }A{p}_j=p_i^{\top }B(B^{-1}A{p}_j)=p_i^{\top }B({\lambda }_j{p}_j)={\lambda }_j{p}_i^{\top }B{p}_j={\lambda }_j{\delta }_{ij},$$

so $P^{\top }AP=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$. The matrix $P$ is invertible because its columns are $B$-orthonormal, hence linearly independent (a dependence $\sum c_k{p}_k=0$ paired in $⟨\cdot ,\cdot {⟩}_B$ against $p_j$ forces $c_j=0$).

Finally, the ${\lambda }_k$ are the generalised eigenvalues: from $P^{\top }BP=I$ and $P^{\top }AP=\Lambda$ one gets $A=P^{-\top }\Lambda P^{-1}$ and $B=P^{-\top }{P}^{-1}$, so

$$\det (A-\lambda B)=\det (P^{-\top }(\Lambda -\lambda I)P^{-1})=\det (P{)}^{-2}\det (\Lambda -\lambda I)=\det (P{)}^{-2}\prod _k({\lambda }_k-\lambda ).$$

The roots are precisely ${\lambda }_1,\dots ,{\lambda }_n$, all real, and ordering the eigenbasis by decreasing eigenvalue gives ${\lambda }_1\ge \cdots \ge {\lambda }_n$. $\square$

Bridge. This theorem builds toward the small-oscillation theory of the Advanced results, where $A$ is the stiffness form, $B$ the mass form, and the generalised eigenpairs become normal modes with frequencies ${\omega }_k^2={\lambda }_k$; the same congruence reduction appears again in 01.01.15 as the metric-signature normalisation that $B$-orthonormalisation performs, since $P^{\top }BP=I$ is Sylvester's positive-definite case made explicit. The foundational reason the construction works is that a positive-definite $B$ is an inner product in disguise, so $B^{-1}A$ becomes self-adjoint and the spectral theorem 01.01.13 applies verbatim; this is exactly the move that converts a two-form problem into a one-operator problem. The generalised Rayleigh quotient $\rho (v)=(v^{\top }Av)/(v^{\top }Bv)$ generalises the ordinary Rayleigh quotient of 01.01.14, with its critical values the ${\lambda }_k$. Putting these together, simultaneous diagonalisation is the statement that one congruence rounds out $B$ to the identity and squares up $A$ to a diagonal at the same time.

Exercises Intermediate+

Lean formalization Intermediate+

Mathlib carries the finite-dimensional spectral theorem and positive-definite bilinear forms, but not the assembled simultaneous-congruence-diagonalisation theorem for an ordered pair $(A,B)$ with $B\succ 0$. The readable view of the intended statements:

import Mathlib.Analysis.InnerProductSpace.Spectrum
import Mathlib.LinearAlgebra.Matrix.PosDef

open Matrix

variable {n : Type*} [Fintype n] [DecidableEq n]

/-- Simultaneous diagonalisation by congruence of a symmetric pair with the
second matrix positive-definite. NOT a single named Mathlib theorem. -/
theorem simultaneous_diagonalisation
    (A B : Matrix n n ℝ) (hA : A.IsSymm) (hB : B.PosDef) :
    ∃ (P : Matrix n n ℝ) (lam : n → ℝ),
      P.det ≠ 0 ∧
      Pᵀ * B * P = 1 ∧
      Pᵀ * A * P = Matrix.diagonal lam := by
  sorry
  -- Cholesky B = Lᵀ * L (Matrix.PosDef.cholesky-style); diagonalise the
  -- symmetric C = L⁻¹ * A * L⁻ᵀ via the spectral theorem; pull back.

/-- The generalised eigenvalues are the roots of det(A - λ B); equivalently the
ordinary eigenvalues of L⁻¹ A L⁻ᵀ. NOT packaged in Mathlib. -/
theorem generalised_eigenvalues_real
    (A B : Matrix n n ℝ) (hA : A.IsSymm) (hB : B.PosDef)
    (lam : ℝ) (v : n → ℝ) (hv : v ≠ 0)
    (h : A.mulVec v = lam • B.mulVec v) :
    True := by
  trivial  -- placeholder: reality follows from self-adjointness of B⁻¹A in ⟨·,·⟩_B

The proof gap to a clean contribution: (i) expose a Cholesky factorisation B = Lᵀ * L from Matrix.PosDef with L lower-triangular invertible; (ii) transport the finite-dimensional spectral theorem LinearMap.IsSymmetric.eigenvectorBasis to the symmetric C = L⁻¹ A L⁻ᵀ; (iii) pull the eigenbasis back through L⁻ᵀ to obtain P with Pᵀ B P = 1 and Pᵀ A P diagonal, recording the $B$-orthogonality. Each step is reachable from current Mathlib, but the assembled simultaneous_diagonalisation and a generalisedEigenvalue API are not present as named results, and the singular-pencil Weierstrass-Kronecker form is entirely absent.

Advanced results Master

Theorem (small oscillations; Lagrange 1788 ^{[source pending]}). Let a mechanical system with generalised coordinates $q\in {\mathbb{R}}^n$ have kinetic energy $T=\frac{1}{2}{\dot{q}}^{\top }B\dot{q}$ and potential energy $V=\frac{1}{2}{q}^{\top }Aq$ near a stable equilibrium, with $B\succ 0$ the mass form and $A=A^{\top }$ the stiffness form. The linearised equations of motion $B\ddot{q}+Aq=0$ decouple in the simultaneous-diagonalisation coordinates $q=Py$ into $n$ independent harmonic oscillators

$${\ddot{y}}_k+{\lambda }_k{y}_k=0,k=1,\dots ,n,$$

where the ${\lambda }_k$ are the generalised eigenvalues of $(A,B)$. Each $y_k$ is a normal coordinate; the $k$-th normal mode oscillates at angular frequency ${\omega }_k=\sqrt{{\lambda }_k}$ when ${\lambda }_k>0$, and the equilibrium is stable exactly when every ${\lambda }_k>0$, that is when $A\succ 0$.

The substitution $q=Py$ with $P^{\top }BP=I$, $P^{\top }AP=\Lambda$ sends $T$ to $\frac{1}{2}{\dot{y}}^{\top }\dot{y}$ and $V$ to $\frac{1}{2}{y}^{\top }\Lambda y=\frac{1}{2}{\sum }_k{\lambda }_k{y}_k^2$, so the Lagrangian separates and the Euler-Lagrange equations become the decoupled set above. This is the mechanical content of simultaneous diagonalisation: the normal modes are the generalised eigenvectors, and the normal frequencies squared are the generalised eigenvalues.

Theorem (generalised Rayleigh quotient and min-max; cf. 01.01.14). With ${\lambda }_1\ge \cdots \ge {\lambda }_n$ the generalised eigenvalues of $(A,B)$, $B\succ 0$, the generalised Rayleigh quotient $\rho (v)=(v^{\top }Av)/(v^{\top }Bv)$ has range $[{\lambda }_n,{\lambda }_1]$ and

$${\lambda }_k=\underset{\dim S=k}{\max }\underset{0\ne v\in S}{\min }\rho (v)=\underset{\dim S=n-k+1}{\min }\underset{0\ne v\in S}{\max }\rho (v).$$

The extremes ${\lambda }_1={\max }_v\rho (v)$ and ${\lambda }_n={\min }_v\rho (v)$ are attained on the corresponding generalised eigenvectors.

The Cholesky substitution $w=L^{\top }v$ identifies $\rho$ with the ordinary Rayleigh quotient of $C=L^{-1}A{L}^{-\top }$, so every result of 01.01.14 transports: the critical points of $\rho$ are the generalised eigenvectors, and the recursive deflation ${\lambda }_{k+1}={\max }_{v{\perp }_B{p}_1,\dots ,p_k}\rho (v)$ uses $B$-orthogonality rather than ordinary orthogonality.

Theorem (Weierstrass-Kronecker canonical form of a pencil; Weierstrass 1868 ^{[source pending]}; Kronecker 1890 ^{[source pending]}). Let $A-\lambda B$ be a pencil of real (or complex) symmetric matrices. If the pencil is regular ($\det (A-\lambda B)\not\equiv 0$), it is equivalent under strict congruence to a direct sum of blocks indexed by its elementary divisors $(\lambda -{\lambda }_i{)}^{m_i}$ and infinite elementary divisors ${\mu }^m$ (poles at $\lambda =\infty$, present when $B$ is singular). If the pencil is singular ($\det (A-\lambda B)\equiv 0$), additional Kronecker minimal-index blocks $L_{\epsilon },L_{\eta }^{\top }$ appear, encoding the row and column minimal indices of the singular part.

When $B\succ 0$ the pencil is regular with $n$ finite linear elementary divisors $(\lambda -{\lambda }_i)$, no infinite divisors, and no minimal indices, which is the diagonalisable case of the definite theorem. The general theory is what governs the failure modes: a defective regular pencil (repeated ${\lambda }_i$ with $m_i>1$) is not diagonalisable, and a singular pencil arises when $A$ and $B$ share a common null direction. The QZ algorithm of Moler and Stewart computes the generalised Schur form that realises this classification numerically.

Theorem (Williamson's symplectic normal form; pointer). Let $A\succ 0$ be a positive-definite real symmetric $2n\times 2n$ matrix and $J=\left(\begin{array}{cc}0& I_n\\ -I_n& 0\end{array}\right)$ the standard symplectic form. There is a symplectic $S$ (meaning $S^{\top }JS=J$) with

$$S^{\top }AS=\left(\begin{array}{cc}D& 0\\ 0& D\end{array}\right),D=\mathrm{diag}(d_1,\dots ,d_n),d_k>0,$$

and the $d_k$, the symplectic eigenvalues, are the positive parts of the eigenvalues of $iJA$.

This is the simultaneous-normal-form theorem when the second structure is a symplectic form rather than a positive-definite form: instead of congruence by an arbitrary $B$-orthonormal $P$, the allowed transformations are symplectic, and the invariants are the symplectic eigenvalues. Williamson's theorem is the linear-algebra backbone of Gaussian quantum information and the metaplectic representation, exactly as the definite case underpins classical normal modes.

Synthesis. Simultaneous diagonalisation of two real symmetric forms, one positive-definite, is the spectral theorem 01.01.13 read in the inner product that the definite form supplies. A positive-definite $B$ is an inner product $⟨u,v{⟩}_B=u^{\top }Bv$; in that inner product the operator $B^{-1}A$ is self-adjoint, so it has a $B$-orthonormal eigenbasis, and the matrix $P$ of that basis achieves $P^{\top }BP=I$ and $P^{\top }AP=\Lambda$ at once. The diagonal entries are the generalised eigenvalues, the roots of $\det (A-\lambda B)$, equivalently the ordinary eigenvalues of the Cholesky-reduced $C=L^{-1}A{L}^{-\top }$, and the generalised Rayleigh quotient $\rho$ carries the entire Courant-Fischer min-max apparatus of 01.01.14 across the substitution $w=L^{\top }v$. The transformation is congruence, not similarity, which is exactly why the ${\lambda }_k$ are pencil invariants rather than eigenvalues of $A$, and why Sylvester's law of inertia rather than the characteristic polynomial of $A$ governs the signs.

Putting these together, the mechanical reading is Lagrange's normal-mode decomposition, with $A$ the stiffness, $B$ the mass, normal modes the generalised eigenvectors, and normal frequencies ${\omega }_k^2={\lambda }_k$; the definiteness of $B$ is the kinetic-energy positivity that makes the theory work, and its loss pushes the problem into the Weierstrass-Kronecker classification of general pencils. The same template, with the second structure a symplectic form, is Williamson's theorem and its symplectic eigenvalues. Across all of these the central insight is that one congruence rounds one form to the identity and squares the other to a diagonal, converting a coupled two-form problem into an uncoupled spectrum.

Full proof set Master

Proposition (existence and uniqueness up to $B$-orthogonal mixing). Let $A,B$ be real symmetric with $B\succ 0$. A simultaneous congruence diagonalisation $P^{\top }BP=I$, $P^{\top }AP=\Lambda$ exists; the diagonal $\Lambda$ is unique up to permutation, and within each generalised eigenspace the choice of $B$-orthonormal eigenvectors is free up to a $B$-orthogonal transformation.

Proof. Existence is the Key theorem: $T=B^{-1}A$ is self-adjoint for $⟨\cdot ,\cdot {⟩}_B$, hence has a $B$-orthonormal eigenbasis, whose matrix $P$ satisfies the two identities. For the diagonal's uniqueness, suppose $P_1,P_2$ both diagonalise the pair, giving ${\Lambda }_1,{\Lambda }_2$. From $P_i^{\top }B{P}_i=I$ and $P_i^{\top }A{P}_i={\Lambda }_i$, the pencil factors as $A-\lambda B=P_i^{-\top }({\Lambda }_i-\lambda I)P_i^{-1}$, so $\det (A-\lambda B)=\det (P_i{)}^{-2}{\prod }_k({\lambda }_k^{(i)}-\lambda )$. The roots of $\det (A-\lambda B)$ are basis-independent, so the multiset $\{{\lambda }_k^{(1)}\}$ equals $\{{\lambda }_k^{(2)}\}$, giving ${\Lambda }_1={\Lambda }_2$ up to permutation.

For the eigenvector freedom, fix a generalised eigenvalue $\mu$ with generalised eigenspace $E_{\mu }=\ker (A-\mu B)$. If $\{p_j\}$ and $\{p_j^{\prime }\}$ are two $B$-orthonormal bases of $E_{\mu }$, the change-of-basis matrix $Q$ between them satisfies $Q^{\top }Q=I$ because both are $B$-orthonormal: ${\delta }_{ij}=⟨p_i^{\prime },p_j^{\prime }{⟩}_B={\sum }_{k,l}{Q}_{ki}{Q}_{lj}⟨p_k,p_l{⟩}_B=(Q^{\top }Q{)}_{ij}$. So $Q$ is orthogonal, and the choice within $E_{\mu }$ is exactly a $B$-orthogonal mixing. $\square$

Proposition (inertia of the pencil and signs of the generalised eigenvalues). Let $A,B$ be real symmetric with $B\succ 0$, and let $\Lambda =\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$ be the diagonal of generalised eigenvalues. The number of positive, negative, and zero ${\lambda }_k$ equals the inertia $(n_{+},n_{-},n_0)$ of $A$, the Sylvester signature of the form $A$ itself.

Proof. By the Key theorem there is an invertible $P$ with $A=P^{-\top }\Lambda P^{-1}$, so $A$ and $\Lambda$ are congruent. Sylvester's law of inertia ^{[Sylvester 1852]} states that the numbers of positive, negative, and zero diagonal entries of a real symmetric matrix are invariant under real congruence. Applying it to the congruence $A\cong \Lambda$, the inertia of $A$ equals the inertia of $\Lambda$, which counts the signs of the ${\lambda }_k$. In particular the equilibrium of the small-oscillation system is stable ($A\succ 0$) precisely when all ${\lambda }_k>0$. $\square$

Proposition (positivity of $B^{-1}A$'s eigenvalues forces simultaneous definiteness). If $A,B$ are real symmetric with $B\succ 0$, then $A\succ 0$ if and only if every generalised eigenvalue ${\lambda }_k>0$, and $A⪰0$ if and only if every ${\lambda }_k\ge 0$.

Proof. Diagonalise simultaneously, $A=P^{-\top }\Lambda P^{-1}$, $B=P^{-\top }{P}^{-1}$. For $v\ne 0$, set $y=P^{-1}v\ne 0$; then $v^{\top }Av=y^{\top }\Lambda y={\sum }_k{\lambda }_k{y}_k^2$ and $v^{\top }Bv={\sum }_k{y}_k^2>0$. If all ${\lambda }_k>0$, then $v^{\top }Av={\sum }_k{\lambda }_k{y}_k^2>0$ for every $v\ne 0$, so $A\succ 0$. Conversely, if some ${\lambda }_j\le 0$, take $v=P{e}_j$ (so $y=e_j$); then $v^{\top }Av={\lambda }_j\le 0$, so $A$ is not positive-definite. The semidefinite statement follows by replacing the strict inequalities throughout. $\square$

Proposition (Cholesky reduction realises the diagonalising congruence). Let $B=L{L}^{\top }$ be the Cholesky factorisation ($L$ lower-triangular, invertible) and let $C=L^{-1}A{L}^{-\top }$, a real symmetric matrix with orthogonal eigendecomposition $C=U\Lambda U^{\top }$, $U^{\top }U=I$. Then $P=L^{-\top }U$ satisfies $P^{\top }BP=I$ and $P^{\top }AP=\Lambda$.

Proof. Compute the $B$-form: $P^{\top }BP=U^{\top }{L}^{-1}(L{L}^{\top })L^{-\top }U=U^{\top }(L^{-1}L)(L^{\top }{L}^{-\top })U=U^{\top }U=I$. Compute the $A$-form: $P^{\top }AP=U^{\top }{L}^{-1}A{L}^{-\top }U=U^{\top }CU=U^{\top }(U\Lambda U^{\top })U=\Lambda$, using $U^{\top }U=I$ twice. The eigenvalues in $\Lambda$ are the eigenvalues of $C$, which by the substitution $w=L^{\top }v$ are the generalised eigenvalues of $(A,B)$. This gives an explicit, numerically standard route to $P$: one Cholesky factorisation and one symmetric eigendecomposition. $\square$

Connections Master

The construction is the spectral theorem 01.01.13 applied in the inner product supplied by the positive-definite form: where the spectral theorem diagonalises a self-adjoint operator in a fixed inner product, this unit chooses the inner product $⟨u,v{⟩}_B=u^{\top }Bv$ so that $B^{-1}A$ becomes self-adjoint, and the resulting eigenbasis diagonalises both forms. The reality of the generalised eigenvalues and the $B$-orthogonality of eigenvectors are the spectral theorem's reality and orthogonality conclusions, transported.

The generalised Rayleigh quotient $\rho (v)=(v^{\top }Av)/(v^{\top }Bv)$ specialises the Rayleigh quotient and Courant-Fischer min-max of 01.01.14 through the Cholesky substitution $w=L^{\top }v$, which turns $\rho$ into the ordinary Rayleigh quotient of $C=L^{-1}A{L}^{-\top }$; every extremal and min-max statement there carries over, now with $B$-orthogonality replacing ordinary orthogonality in the deflation.

The congruence transformation $X\mapsto P^{\top }XP$ is the bilinear-form change-of-variables law of 01.01.17, and the sign count of the diagonalised pair is governed by Sylvester's law of inertia from 01.01.15: the generalised eigenvalues inherit the signature of $A$, so the quadratic-form classification by inertia is exactly what determines stability of the associated mechanical equilibrium.

The same template recurs when the second form is symplectic rather than positive-definite, giving Williamson's normal form and the symplectic eigenvalues that organise Gaussian states in quantum information 12.07.03; and the small-oscillation reading feeds the normal-mode analysis of coupled oscillators and lattice vibrations in classical and condensed-matter physics, where the generalised eigenvectors are the phonon polarisations and ${\omega }_k^2={\lambda }_k$ are the squared mode frequencies.

Historical & philosophical context Master

The simultaneous reduction of two quadratic forms grew out of mechanics before it became a chapter of linear algebra. Lagrange, in the Mécanique analytique (1788), analysed the small oscillations of a system about a stable equilibrium by writing the kinetic and potential energies as two quadratic forms and seeking coordinates in which both became sums of squares, so that the equations of motion separated into independent harmonic oscillators; the determinant condition he reached is the secular equation $\det (A-\lambda B)=0$ that names the squared normal frequencies ^{[Lagrange 1788]}. Sylvester's law of inertia (1852) supplied the invariant that the reduction respects: the signature of a real quadratic form is unchanged by real congruence, so the signs of the generalised eigenvalues are intrinsic ^{[Sylvester 1852]}.

The general theory of a pencil of forms is due to Weierstrass and Kronecker. Weierstrass (1868) classified a regular pencil $A-\lambda B$ by its elementary divisors, settling exactly when the pair is simultaneously diagonalisable and when it is defective ^{[Weierstrass 1868]}. Kronecker (1890) completed the picture for singular pencils, introducing the minimal indices that measure the singular part beyond the Weierstrass divisors ^{[Kronecker 1890]}. The positive-definite case singled out in this unit is the benign corner of that theory, where all elementary divisors are finite and linear and the pencil diagonalises by a real congruence. Shilov's Linear Algebra presents the definite case as the simultaneous-reduction theorem, and Parlett's The Symmetric Eigenvalue Problem (1980) developed the numerical side, the Cholesky reduction and the conditioning of the symmetric-definite generalised eigenproblem that underlies modern eigensolvers ^{[Parlett 1980]}.

Bibliography Master

@book{Lagrange1788,
  author    = {Lagrange, Joseph-Louis},
  title     = {M{\'e}canique analytique},
  publisher = {Veuve Desaint},
  address   = {Paris},
  year      = {1788}
}

@article{Sylvester1852,
  author  = {Sylvester, James Joseph},
  title   = {A demonstration of the theorem that every homogeneous quadratic polynomial is reducible by real orthogonal substitutions to the form of a sum of positive and negative squares},
  journal = {Philosophical Magazine (4th series)},
  volume  = {4},
  number  = {23},
  year    = {1852},
  pages   = {138--142}
}

@article{Weierstrass1868,
  author  = {Weierstrass, Karl},
  title   = {Zur Theorie der bilinearen und quadratischen Formen},
  journal = {Monatsberichte der K{\"o}niglich Preussischen Akademie der Wissenschaften zu Berlin},
  year    = {1868},
  pages   = {310--338}
}

@article{Kronecker1890,
  author  = {Kronecker, Leopold},
  title   = {Algebraische Reduction der Schaaren bilinearer Formen},
  journal = {Sitzungsberichte der K{\"o}niglich Preussischen Akademie der Wissenschaften zu Berlin},
  year    = {1890},
  pages   = {763--776}
}

@book{Gantmacher1959,
  author    = {Gantmacher, Felix R.},
  title     = {The Theory of Matrices, Volume I},
  publisher = {Chelsea},
  address   = {New York},
  year      = {1959}
}

@book{Parlett1998,
  author    = {Parlett, Beresford N.},
  title     = {The Symmetric Eigenvalue Problem},
  publisher = {SIAM},
  series    = {Classics in Applied Mathematics},
  year      = {1998},
  note      = {Originally published Prentice-Hall, 1980}
}

@book{GolubVanLoan2013,
  author    = {Golub, Gene H. and Van Loan, Charles F.},
  title     = {Matrix Computations},
  edition   = {4th},
  publisher = {Johns Hopkins University Press},
  year      = {2013}
}

@book{Shilov1977,
  author    = {Shilov, Georgi E.},
  title     = {Linear Algebra},
  publisher = {Dover Publications},
  address   = {New York},
  year      = {1977},
  note      = {Translation of the 1971 Russian edition, transl. R. A. Silverman}
}

@book{Arnold1989,
  author    = {Arnold, Vladimir I.},
  title     = {Mathematical Methods of Classical Mechanics},
  edition   = {2nd},
  publisher = {Springer},
  series    = {Graduate Texts in Mathematics},
  volume    = {60},
  year      = {1989}
}

@article{MolerStewart1973,
  author  = {Moler, Cleve B. and Stewart, G. W.},
  title   = {An algorithm for generalized matrix eigenvalue problems},
  journal = {SIAM Journal on Numerical Analysis},
  volume  = {10},
  number  = {2},
  year    = {1973},
  pages   = {241--256}
}

@article{Williamson1936,
  author  = {Williamson, John},
  title   = {On the algebraic problem concerning the normal forms of linear dynamical systems},
  journal = {American Journal of Mathematics},
  volume  = {58},
  number  = {1},
  year    = {1936},
  pages   = {141--163}
}