Engel's theorem + Lie's theorem

Intuition [Beginner]

Engel's theorem captures a clean fact about Lie algebras acting on vector spaces. If every element of the Lie algebra acts as a nilpotent transformation — meaning repeated application sends every vector to zero — then there is a single non-zero vector killed by every element at once. Moreover, there is a basis of the vector space in which every representing matrix is strictly upper triangular: all entries on and below the diagonal are zero.

Lie's theorem covers a broader class called solvable Lie algebras. These are algebras whose commutator brackets shrink to zero after finitely many steps. Over the complex numbers, Lie's theorem guarantees a common eigenvector: a non-zero vector that each element of the algebra scales by some number. The representing matrices become upper triangular — the diagonal entries are now allowed to be non-zero, but everything below the diagonal is still zero.

Every finite-dimensional Lie algebra decomposes into a solvable part and a semisimple part. Engel's and Lie's theorems handle the solvable piece; the semisimple piece is managed by the Killing form and root systems. Without these two theorems, the classification of Lie algebras does not start.

Visual [Beginner]

A picture showing a three-dimensional vector space with a nested flag of subspaces: a line inside a plane inside the full space. Arrows labelled by Lie algebra elements map each subspace into the previous level, illustrating Engel's theorem. A second panel shows Lie's theorem with diagonal arrows indicating eigenvectors rather than zero vectors.

The picture captures the core mechanism: Engel's theorem gives a flag where the action strictly decreases the level, and Lie's theorem gives a flag where the action preserves each level modulo the previous one.

Worked example [Beginner]

Consider the Heisenberg Lie algebra acting on ${\mathbb{C}}^3$. This algebra has three basis elements represented as $3\times 3$ matrices:

$$P=\left(\begin{array}{ccc}0& 1& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right),Q=\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 1\\ 0& 0& 0\end{array}\right),Z=\left(\begin{array}{ccc}0& 0& 1\\ 0& 0& 0\\ 0& 0& 0\end{array}\right)$$

with bracket $[P,Q]=Z$ and $Z$ central.

Step 1. Each element is nilpotent. The matrix $P$ satisfies $P^2=0$ (multiplying $P$ by itself gives the zero matrix). The same holds for $Q$ and $Z$: both square to zero.

Step 2. The vector $e_1=(1,0,0)$ is killed by every element: $P{e}_1=0$, $Q{e}_1=0$, and $Z{e}_1=0$. This is the common zero vector guaranteed by Engel's theorem.

Step 3. In the reordered basis $\{e_1,e_3,e_2\}$, every matrix becomes strictly upper triangular. Each element maps basis vectors to combinations of earlier ones only.

What this tells us: individual nilpotency of $P$, $Q$, $Z$ forces a simultaneous strict upper-triangular form — exactly what Engel's theorem guarantees.

Check your understanding [Beginner]

Formal definition [Intermediate+]

A nilpotent Lie algebra is one whose lower central series terminates at zero. Define ${\mathfrak{g}}^0=\mathfrak{g}$ and ${\mathfrak{g}}^{i+1}=[\mathfrak{g},{\mathfrak{g}}^i]$ (the subspace spanned by all brackets $[X,Y]$ with $X\in \mathfrak{g}$, $Y\in {\mathfrak{g}}^i$). The Lie algebra $\mathfrak{g}$ is nilpotent (of class $\le n$) if ${\mathfrak{g}}^n=0$. The chain is $$\mathfrak{g}={\mathfrak{g}}^0\supseteq {\mathfrak{g}}^1\supseteq {\mathfrak{g}}^2\supseteq \cdots .$$

A solvable Lie algebra is one whose derived series terminates at zero. Define ${\mathfrak{g}}^{(0)}=\mathfrak{g}$ and ${\mathfrak{g}}^{(i+1)}=[{\mathfrak{g}}^{(i)},{\mathfrak{g}}^{(i)}]$. The Lie algebra $\mathfrak{g}$ is solvable (of derived length $\le n$) if ${\mathfrak{g}}^{(n)}=0$. The chain is $$\mathfrak{g}={\mathfrak{g}}^{(0)}\supseteq {\mathfrak{g}}^{(1)}\supseteq {\mathfrak{g}}^{(2)}\supseteq \cdots .$$

The containment ${\mathfrak{g}}^{(i)}\subseteq {\mathfrak{g}}^i$ holds for all $i$ (proved by induction: ${\mathfrak{g}}^{(1)}=[\mathfrak{g},\mathfrak{g}]={\mathfrak{g}}^1$, and if ${\mathfrak{g}}^{(i)}\subseteq {\mathfrak{g}}^i$, then ${\mathfrak{g}}^{(i+1)}=[{\mathfrak{g}}^{(i)},{\mathfrak{g}}^{(i)}]\subseteq [\mathfrak{g},{\mathfrak{g}}^i]={\mathfrak{g}}^{i+1}$). Hence nilpotent implies solvable.

A linear transformation $T$ on a finite-dimensional vector space $V$ is nilpotent if $T^k=0$ for some positive integer $k$. The smallest such $k$ is the nilpotency index.

Counterexamples to common slips

• Nilpotent is stronger than solvable. The Lie algebra ${\mathfrak{b}}_2$ of traceless upper triangular $2\times 2$ matrices is solvable (${\mathfrak{b}}_2^{(1)}=\mathbb{C}E$, ${\mathfrak{b}}_2^{(2)}=0$) but not nilpotent (${\mathfrak{b}}_2^1=\mathbb{C}E$, ${\mathfrak{b}}_2^2=\mathbb{C}E$, and the lower central series stabilises without reaching zero).
• Engel's theorem is about representations, not just the algebra. Engel's theorem concerns a Lie algebra $\mathfrak{g}$ acting on a vector space $V$ via $\rho :\mathfrak{g}\to \mathfrak{gl}(V)$. The conclusion is about the simultaneous action, not the internal bracket structure of $\mathfrak{g}$.
• Lie's theorem requires algebraically closed ground field. Over $\mathbb{R}$ the theorem fails: the two-dimensional rotation Lie algebra acting on ${\mathbb{R}}^2$ is abelian (hence solvable) but the rotation matrix has no real eigenvectors.

Key theorem with proof [Intermediate+]

Theorem (Engel). Let $\mathfrak{g}$ be a Lie algebra and $\rho :\mathfrak{g}\to \mathfrak{gl}(V)$ a finite-dimensional representation over $\mathbb{C}$. If $\rho (X)$ is nilpotent for every $X\in \mathfrak{g}$, then there exists $0\ne v\in V$ such that $\rho (X)v=0$ for all $X\in \mathfrak{g}$.

Proof. The proof is by induction on $\dim \mathfrak{g}$.

Base case. If $\dim \mathfrak{g}=0$ the statement is vacuous. If $\dim \mathfrak{g}=1$, say $\mathfrak{g}=\mathbb{C}X$, the operator $\rho (X)$ is nilpotent on the non-zero space $V$, so $\ker \rho (X)\ne 0$ and any non-zero element of the kernel is a common zero vector.

Inductive step. Assume the theorem for all Lie algebras of dimension strictly less than $n=\dim \mathfrak{g}\ge 2$.

Stage 1: Codimension-one ideal. Let $\mathfrak{m}$ be a maximal proper subalgebra of $\mathfrak{g}$. The adjoint action of $\mathfrak{m}$ on the quotient $\mathfrak{g}/\mathfrak{m}$ defines a representation $\overline{\mathrm{ad}}:\mathfrak{m}\to \mathfrak{gl}(\mathfrak{g}/\mathfrak{m})$. Each ${\overline{\mathrm{ad}}}_X$ for $X\in \mathfrak{m}$ is nilpotent (the quotient of a nilpotent operator is nilpotent). Since $\dim \mathfrak{m}<n$, the induction hypothesis applies: there exists $\bar{Y}\ne 0$ in $\mathfrak{g}/\mathfrak{m}$ with ${\overline{\mathrm{ad}}}_X(\bar{Y})=0$ for all $X\in \mathfrak{m}$. Lifting $\bar{Y}$ to $Y\in \mathfrak{g}$, this says $[X,Y]\in \mathfrak{m}$ for all $X\in \mathfrak{m}$. The subspace $\mathfrak{m}+\mathbb{C}Y$ is a subalgebra of $\mathfrak{g}$ strictly containing $\mathfrak{m}$, so by maximality $\mathfrak{m}+\mathbb{C}Y=\mathfrak{g}$. Thus $\mathfrak{m}$ has codimension one and is an ideal of $\mathfrak{g}$ (since $[\mathfrak{m},Y]\subseteq \mathfrak{m}$).

Stage 2: Common zero vector. Write $\mathfrak{g}=\mathfrak{m}\oplus \mathbb{C}Y$. By the induction hypothesis applied to $\rho {|}_{\mathfrak{m}}$, the subspace $$W:=\{w\in V:\rho (X)w=0\text{ for all }X\in \mathfrak{m}\}$$ is non-zero. For any $w\in W$ and $X\in \mathfrak{m}$: $$\rho (X)\rho (Y)w=\rho (Y)\rho (X)w+\rho ([X,Y])w=0+0=0,$$ using $[X,Y]\in \mathfrak{m}$ (ideal) and $w\in W$. So $\rho (Y)W\subseteq W$. The restriction $\rho (Y){|}_W$ is nilpotent (restriction of a nilpotent operator to an invariant subspace is nilpotent) on the non-zero space $W$. A nilpotent operator on a non-zero space has a non-zero kernel: pick $0\ne w_0\in W$ with $\rho (Y)w_0=0$. Then $\rho (X)w_0=0$ for all $X\in \mathfrak{m}$ (since $w_0\in W$) and $\rho (Y)w_0=0$, hence $\rho (X)w_0=0$ for all $X\in \mathfrak{g}$. $\square$

Corollary (Strict upper-triangular form). Under the hypotheses of Engel's theorem, there exists a basis of $V$ in which every $\rho (X)$ is strictly upper triangular.

Proof. By Engel's theorem, choose $0\ne v_1\in V$ with $\rho (\mathfrak{g})v_1=0$. The line $\mathbb{C}{v}_1$ is $\mathfrak{g}$-invariant, and $\rho (\mathfrak{g})$ acts by nilpotent operators on the quotient $V/\mathbb{C}{v}_1$. Induction on $\dim V$ produces a flag $0=V_0\subset V_1=\mathbb{C}{v}_1\subset V_2\subset \cdots \subset V_n=V$ with $\dim V_i=i$ and $\rho (\mathfrak{g})V_i\subseteq V_{i-1}$. In any basis adapted to this flag, every $\rho (X)$ is strictly upper triangular. $\square$

Bridge. Engel's theorem builds toward 07.06.11 where the ${\mathfrak{sl}}_2\mathbb{C}$ ladder starts from a highest-weight vector killed by the raising operator — a nilpotent element to which Engel applies in a single step. The foundational reason Engel's theorem works is the codimension-one ideal structure: the maximal subalgebra $\mathfrak{m}$ drops the dimension by one, and the nilpotency on the quotient feeds back a common zero vector. This is exactly the same inductive engine that drives Lie's theorem in the solvable case, and appears again in 07.06.03 where root-space decomposition uses reductions relative to the Cartan subalgebra. The central insight is that a structural hypothesis on individual elements propagates to a simultaneous structural conclusion across the entire algebra, and the bridge is the recognition that both Engel's and Lie's theorems are flag-construction results whose common mechanism is inductive descent through codimension-one ideals.

Exercises [Intermediate+]

Advanced results [Master]

Theorem 1 (Lie's theorem — flag form). Let $\mathfrak{g}$ be a solvable Lie algebra over $\mathbb{C}$ and $\rho :\mathfrak{g}\to \mathfrak{gl}(V)$ a finite-dimensional representation. There exists a flag $0=V_0\subset V_1\subset \cdots \subset V_n=V$ with $\dim V_i=i$ and $\rho (\mathfrak{g})V_i\subseteq V_i$ for each $i$. Equivalently, there is a basis of $V$ in which every $\rho (X)$ is upper triangular.

The diagonal entries of each $\rho (X)$ are linear functionals ${\lambda }_1,\dots ,{\lambda }_n:\mathfrak{g}\to \mathbb{C}$, called the weights of the representation (not to be confused with weight vectors for a Cartan subalgebra).

Theorem 2 (Borel subalgebra). A Borel subalgebra of a complex semisimple Lie algebra $\mathfrak{g}$ is a maximal solvable subalgebra. Every Borel subalgebra has the form $\mathfrak{h}\oplus {⨁}_{\alpha \in {\Phi }^{+}}{\mathfrak{g}}_{\alpha }$ for some choice of Cartan subalgebra $\mathfrak{h}$ and positive roots ${\Phi }^{+}$. Any two Borel subalgebras are conjugate under the adjoint group of $\mathfrak{g}$.

Theorem 3 (Lie's theorem — vanishing of commutator traces). If $\mathfrak{g}$ is solvable and $\rho :\mathfrak{g}\to \mathfrak{gl}(V)$ is a finite-dimensional representation, then $\mathrm{tr}(\rho ([X,Y])\rho (Z))=0$ for all $X,Y,Z\in \mathfrak{g}$. In particular, the Killing form $B(X,Y)=\mathrm{tr}({\mathrm{ad}}_X{\mathrm{ad}}_Y)$ satisfies $B([\mathfrak{g},\mathfrak{g}],\mathfrak{g})=0$ when $\mathfrak{g}$ is solvable.

Theorem 4 (Engel — adjoint form). A finite-dimensional Lie algebra $\mathfrak{g}$ is nilpotent if and only if ${\mathrm{ad}}_X$ is nilpotent for every $X\in \mathfrak{g}$.

Theorem 5 (Flag variety). The set $\mathcal{B}$ of Borel subalgebras of a semisimple Lie algebra $\mathfrak{g}$ has the structure of a smooth projective variety, called the flag variety. The Borel-Weil theorem 07.06.09 identifies the irreducible representations of $\mathfrak{g}$ with the cohomology of line bundles on $\mathcal{B}$.

Theorem 6 (Nilpotent radical). The nilradical (largest nilpotent ideal) of a solvable Lie algebra $\mathfrak{g}$ is the set of elements $X\in \mathfrak{g}$ such that ${\mathrm{ad}}_X$ is nilpotent. It contains $[\mathfrak{g},\mathfrak{g}]$ and is nilpotent by Engel's theorem applied to the adjoint representation.

Synthesis. The foundational reason Engel's and Lie's theorems work is the codimension-one ideal structure of nilpotent and solvable Lie algebras, which enables inductive descent on dimension. The central insight is that a structural hypothesis on individual elements — nilpotency for Engel, solvability for Lie — propagates to a simultaneous structural conclusion across the entire algebra and its representations. This is exactly the content that identifies the solvable and semisimple worlds as complementary: the Killing form is non-degenerate on the semisimple piece and vanishes on the solvable radical, and the bridge is between the flag constructions of this unit on one side and the root-space decomposition of 07.06.03 on the other. Putting these together, every finite-dimensional Lie algebra is a semidirect product of its radical and a semisimple subalgebra, and the pattern recurs in 07.06.10 where the Casimir detects the semisimple part by acting as distinct scalars on distinct irreducibles. The generalises direction runs from these solvable theorems toward the full Levi decomposition: the radical is solvable (hence amenable to Lie's theorem), the Levi factor is semisimple (hence controlled by root systems and the Casimir), and identifying the radical with the kernel of the Killing form is the structural content of Cartan's criterion.

Full proof set [Master]

Proposition 1 (Lie's theorem — trace vanishing). If $\mathfrak{g}$ is solvable and $\rho :\mathfrak{g}\to \mathfrak{gl}(V)$ is a finite-dimensional representation over $\mathbb{C}$, then $\mathrm{tr}(\rho ([X,Y])\rho (Z))=0$ for all $X,Y,Z\in \mathfrak{g}$.

Proof. By Lie's theorem, choose a basis of $V$ in which every $\rho (X)$ is upper triangular, with diagonal entries ${\lambda }_1(X),\dots ,{\lambda }_n(X)$. The commutator $\rho ([X,Y])=\rho (X)\rho (Y)-\rho (Y)\rho (X)$ is strictly upper triangular in this basis (the diagonal entries of $\rho (X)\rho (Y)$ are ${\lambda }_i(X){\lambda }_i(Y)$, same as for $\rho (Y)\rho (X)$, so they cancel). A strictly upper triangular matrix has all diagonal entries zero, so $\mathrm{tr}(\rho ([X,Y]))=0$. The product $\rho ([X,Y])\rho (Z)$ is then strictly upper triangular times upper triangular, which is strictly upper triangular (the diagonal entries of the product are zero times ${\lambda }_i(Z)=0$). So $\mathrm{tr}(\rho ([X,Y])\rho (Z))=0$. $\square$

Proposition 2 (Lie's theorem — corollary on Killing form). If $\mathfrak{g}$ is solvable, then $B(\mathfrak{g},[\mathfrak{g},\mathfrak{g}])=0$, where $B$ is the Killing form.

Proof. Apply Proposition 1 with $\rho =\mathrm{ad}$: the Killing form is $B(X,Y)=\mathrm{tr}({\mathrm{ad}}_X{\mathrm{ad}}_Y)$. Setting $Z=X$ gives $B([X,Y],Z)=\mathrm{tr}({\mathrm{ad}}_{[X,Y]}{\mathrm{ad}}_Z)=0$ for all $X,Y,Z\in \mathfrak{g}$. Since the elements of $[\mathfrak{g},\mathfrak{g}]$ are linear combinations of commutators, and $B$ is bilinear, $B(Z,[X,Y])=0$ for all $Z\in \mathfrak{g}$ and $[X,Y]\in [\mathfrak{g},\mathfrak{g}]$. $\square$

Connections [Master]

• Lie algebra representation 07.06.01. The foundational framework for both Engel's and Lie's theorems is the notion of a Lie algebra acting on a vector space via a representation $\rho :\mathfrak{g}\to \mathfrak{gl}(V)$. Engel's theorem is a statement about the nilpotency of such an action; Lie's theorem is a statement about its simultaneous triangularisation. Both are results about the image of $\rho$, not the abstract algebra alone.

• Casimir element 07.06.10. The Casimir element detects the semisimple part of a Lie algebra: on the semisimple quotient, the Killing form is non-degenerate and the Casimir is well-defined. On the solvable radical, the Killing form vanishes by Proposition 2. The complementarity between Engel-Lie (solvable theory) and the Casimir (semisimple theory) is the structural input to the Levi decomposition.

• Representations of ${\mathfrak{sl}}_2\mathbb{C}$ 07.06.11. The Borel subalgebra ${\mathfrak{b}}_2$ of ${\mathfrak{sl}}_2\mathbb{C}$ is solvable, and Lie's theorem applied to ${\mathfrak{b}}_2$ gives the highest-weight vector that initiates the ladder construction. Engel's theorem applied to the nilpotent radical $\mathbb{C}E$ gives the annihilation condition $E{v}_n=0$ at the top of the ladder. The representation theory of ${\mathfrak{sl}}_2\mathbb{C}$ is thus built from one Engel step and one Lie step.

Historical & philosophical context [Master]

Friedrich Engel proved his theorem in the late 1880s during his collaboration with Sophus Lie on the multi-volume Theorie der Transformationsgruppen (Teubner, Leipzig, 1888–1893) ^[pending]. Engel's contribution was the recognition that the nilpotency of every individual element of a Lie algebra of linear transformations forces a simultaneous strict upper-triangular form — the deepest structural result about nilpotent Lie algebras, and one that makes the entire inductive machinery of Lie algebra classification possible.

Sophus Lie had stated the theorem that bears his name as early as 1876 in the context of his theory of continuous transformation groups ^[pending]. Lie's original formulation concerned the reduction of a system of linear partial differential equations to a triangular form; the modern formulation as a result about common eigenvectors of solvable Lie algebras crystallised with the work of Eugenio Elia Levi (1905) and the structural synthesis of Élie Cartan in his 1894 thesis ^[pending], where Cartan used both Engel's and Lie's theorems as tools for his classification of simple Lie algebras. The flag-variety interpretation and the connection to Borel subalgebras came later with Armand Borel's work in the 1950s.

Bibliography [Master]

@book{LieEngel1888,
  author    = {Lie, Sophus and Engel, Friedrich},
  title     = {Theorie der Transformationsgruppen},
  volume    = {I},
  publisher = {B. G. Teubner},
  address   = {Leipzig},
  year      = {1888}
}

@phdthesis{Cartan1894Thesis,
  author    = {Cartan, {\'E}lie},
  title     = {Sur la structure des groupes de transformations finis et continus},
  school    = {Facult{\'e} des Sciences de Paris},
  year      = {1894}
}

@article{Levi1905,
  author  = {Levi, Eugenio Elia},
  title   = {Sulla struttura dei gruppi finiti e continui},
  journal = {Atti della Reale Accademia delle Scienze di Torino},
  volume  = {40},
  year    = {1905},
  pages   = {551--565}
}

@book{HumphreysLieAlgebras,
  author    = {Humphreys, James E.},
  title     = {Introduction to {L}ie Algebras and Representation Theory},
  publisher = {Springer-Verlag},
  series    = {Graduate Texts in Mathematics},
  volume    = {9},
  year      = {1972}
}

@book{FultonHarrisRepTheory,
  author    = {Fulton, William and Harris, Joe},
  title     = {Representation Theory: A First Course},
  publisher = {Springer-Verlag},
  series    = {Graduate Texts in Mathematics},
  volume    = {129},
  year      = {1991}
}

@book{SerreCSLA,
  author    = {Serre, Jean-Pierre},
  title     = {Complex Semisimple {L}ie Algebras},
  publisher = {Springer-Verlag},
  year      = {2001},
  note      = {Translated from the French original (1966)}
}

@book{HallLie,
  author    = {Hall, Brian C.},
  title     = {{L}ie Groups, {L}ie Algebras, and Representations},
  publisher = {Springer},
  edition   = {2},
  series    = {Graduate Texts in Mathematics},
  volume    = {222},
  year      = {2015}
}

@book{BourbakiGroupesAlgebres18,
  author    = {Bourbaki, Nicolas},
  title     = {Groupes et alg{\`e}bres de {L}ie, Chapitres 1--8},
  publisher = {Hermann (orig.) / Springer (reprint)},
  year      = {1972}
}