String Algorithms: KMP, Suffix Arrays, and Aho-Corasick
Anchor (Master): Cormen, Leiserson, Rivest & Stein 2022 Introduction to Algorithms 4e (MIT Press) §31.1-31.4; Gusfield 1997 Algorithms on Strings, Trees, and Sequences (Cambridge) Ch. 1-3 (suffix trees and arrays, LCP arrays, Aho-Corasick as a generalisation of KMP to automata)
Intuition Beginner
You have a long document and want to find every occurrence of the word "abracadabra." The obvious way is to check every starting position: try position 1, then position 2, and so on. Each check compares up to 11 characters. For a document of length , this takes up to comparisons — slow if the pattern or text is long.
The problem is that this naive approach forgets what it has learned. Suppose you are matching "aab" against the text "aaab." You compare positions 1-3: "aaa" vs "aab" — mismatch at position 3. The naive approach starts over at position 2. But you already know the first two characters of the text are "aa," which is also a prefix of "aab." You should not recheck them.
The Knuth-Morris-Pratt (KMP) algorithm captures this insight. It precomputes a "failure function" that tells you, after a mismatch, how many characters you can skip without rechecking. The text is scanned left to right without ever backing up.
Suffix arrays solve a different problem: what if you want to search for many different patterns in the same long text? Instead of scanning the text each time, you sort all suffixes of the text alphabetically. Then searching for a pattern is just binary search among the sorted suffixes.
Aho-Corasick generalises KMP to multiple patterns simultaneously. You build a trie (prefix tree) of all patterns and add "failure links" that work like KMP's failure function. Scanning the text once finds all occurrences of all patterns.
Visual Beginner
KMP failure function for pattern "aabbaab":
| Position q | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|
| P[q] | a | a | b | b | a | a | b |
| pi[q] | 0 | 1 | 0 | 0 | 1 | 2 | 3 |
The failure function gives the length of the longest proper prefix of that is also a suffix. For : "aa" has prefix "a" = suffix "a", so . For : "aabbaa" has longest prefix-suffix "aa", so .
A suffix array for text "banana$":
| Index | Suffix | Sorted position |
|---|---|---|
| 7 | $ | 1 |
| 4 | ana$ | 3 |
| 1 | banana$ | 5 |
| 3 | nana$ | 7 |
Search for "ana": binary search finds it at sorted positions 3-4 (indices 4, 2).
Worked example Beginner
We run KMP on text = "aabababaac" and pattern = "aabaa".
Step 1. Compute failure function for "aabaa":
(single character has no proper prefix-suffix). ("aa": prefix "a" = suffix "a"). ("aab": no prefix = suffix). ("aaba": prefix "a" = suffix "a"). ("aabaa": prefix "aa" = suffix "aa").
Step 2. Match against :
. Compare . . . Compare . . . Compare . . . Compare . Wait, and . Match! .
Let me redo with a clearer trace. = "aabaa", = "aabababaac".
: . Match. . : . Match. . : . Match. . : . Match. . : . Mismatch! Use . .
Now : . Mismatch. Use . .
: . Advance .
: . Match. . : . Mismatch. . . : . . : . . : . . : . Mismatch. . . : . . . : . . End of text.
No matches found in this trace (the pattern "aabaa" does not occur in "aabababaac").
Check your understanding Beginner
Formal definition Intermediate+
Definition (string-matching problem). Given a pattern and a text over an alphabet , find all shifts such that .
Definition (KMP failure function). The failure function (also called the prefix function) for pattern is:
for . By convention, if no such exists.
Theorem (KMP running time). The KMP algorithm computes the failure function in time and finds all occurrences of in in time, for a total of .
Proof sketch. The failure function computation uses an amortised argument: the variable (current prefix length) starts at 0, increases by at most 1 per iteration (to at most ), and the total number of decreases across all iterations is bounded by the total increases, which is at most . So the total work is .
For matching: the text index advances monotonically from 1 to . The pattern index increases by at most 1 per character of text (total increases bounded by ) and decreases by at least 1 per failure-function step (total decreases bounded by total increases). So the total matching work is .
Definition (suffix array). The suffix array of a string is a permutation of such that in lexicographic order.
Definition (LCP array). The longest common prefix array is defined by for .
The LCP array accelerates pattern search in the suffix array: when binary searching for a pattern, the LCP with the current midpoint tells you how many characters have already been matched, avoiding redundant comparisons.
Definition (Aho-Corasick automaton). Given a set of patterns , the Aho-Corasick automaton is a finite automaton (a trie augmented with failure links) that recognises the language .
- The goto function follows trie edges.
- The failure function gives the longest proper suffix of the path to that is also a trie node (analogous to KMP's failure function).
- The output function lists all patterns that end at node .
Theorem (Aho-Corasick running time). The Aho-Corasick automaton can be built in time and searches a text of length in time where is the number of matches.
Counterexamples to common slips
"KMP is always faster than the naive algorithm." For short patterns or random text, the naive algorithm is often faster in practice due to lower constant factors. KMP's advantage is worst-case efficiency on adversarial inputs.
"A suffix array is the same as a suffix tree." A suffix array stores the same information as a suffix tree but in a more compact form ( integers vs tree nodes). Some operations are slower on suffix arrays but the space savings are significant.
"Aho-Corasick is just running KMP multiple times." Aho-Corasick shares computation across patterns. The trie merges common prefixes, and failure links allow all patterns to be tracked simultaneously in a single pass over the text.
Key theorem with proof Intermediate+
Theorem (suffix array construction in ). The suffix array of a string of length can be constructed in time using a prefix-doubling approach.
Proof sketch. Sort suffixes by their first character. Then sort by their first 2 characters (using the sort-by-1 as a tiebreaker). Then by 4, 8, etc. After doubling steps, the sort is by the full suffix. Each step sorts items by a key from a bounded alphabet (pairs of ranks from the previous step), which can be done in using radix sort. Total: .
More precisely, let be the rank of among all length- substrings. Initially, . To compute , sort the pairs . Each sorting step takes with radix sort, and we need steps.
Bridge. String algorithms are the foundational reason pattern matching can be done in linear time: KMP avoids redundant comparisons by exploiting the prefix structure of the pattern, suffix arrays enable indexed search by sorting suffixes, and Aho-Corasick generalises KMP to multiple patterns using automata theory. This builds toward the regular expression engines of 47.01.03, where automata-based matching handles patterns described by regular expressions, and it appears again in 47.01.02, where the NFA-to-DFA conversion parallels the trie-to-automaton construction in Aho-Corasick. The central insight is that the prefix structure of strings — what overlaps exist between prefixes and suffixes — is exactly the information needed to avoid redundant work. Putting these together with the automata theory of 47.01.01 yields a complete picture: string matching is pattern recognition, and the right automaton makes it efficient.
Exercises Intermediate+
Lean formalization Intermediate+
Mathlib has String, List, and Array with basic lemmas but no formalisation of KMP, suffix arrays, or Aho-Corasick. The foundational gap is the absence of the KMP failure function and its correctness proof (showing the algorithm finds all and only valid shifts), the suffix array data structure with its construction and search operations, and the Aho-Corasick automaton with failure links. A formalisation of KMP would require proving that the failure function correctly computes the longest proper prefix-suffix and that the matching phase is complete (finds all occurrences). This unit ships without a lean_module.
Advanced results Master
Three results extend the basic string algorithm toolkit.
Theorem 1 (linear-time suffix array construction). The suffix array of a string of length can be constructed in time using the SA-IS (Suffix Array Induced Sorting) algorithm of Nong, Zhang, and Chan (2011).
SA-IS is based on the observation that suffixes can be classified into L-type and S-type, and that the S-type suffixes form a recursively smaller subproblem. The algorithm recursively sorts the S-type suffixes and then uses induced sorting to place the L-type suffixes. This achieves worst-case time, matching the theoretical bound for suffix tree construction.
Theorem 2 (LCP array from suffix array in linear time). Given the suffix array and the text, the LCP array can be computed in time using Kasai's algorithm.
Kasai's algorithm processes suffixes in text order (not SA order). The key insight is that the LCP of consecutive suffixes in text order differs by at most 1 from the LCP of the previous pair, allowing the computation to proceed in amortised per position [Gusfield Ch. 3].
Theorem 3 (Aho-Corasick as automaton). The Aho-Corasick automaton is a DFA (in the sense of 47.01.01) whose language is . The number of states is and the construction time is .
The Aho-Corasick automaton generalises the KMP automaton (which is the special case ). The failure links play the same role as the KMP failure function: they allow the automaton to "fall back" to the longest matching prefix without reprocessing any text characters. This connects string matching directly to automata theory.
Synthesis. String algorithms are the foundational reason pattern matching can be done efficiently: KMP exploits prefix structure for linear-time matching, suffix arrays enable indexed search by sorting, and Aho-Corasick extends the approach to multiple patterns using automata. The central insight is that the prefix-suffix structure of strings — captured by the failure function, the suffix array ordering, or the trie with failure links — is exactly the information needed to avoid redundant comparisons. This builds toward the regular expression engines of 47.01.03 and the NFA constructions of 47.01.02, where the same automata-theoretic ideas handle more complex patterns. Putting these together yields a layered approach to string processing: exact matching (KMP), indexed search (suffix arrays), and multi-pattern matching (Aho-Corasick), each building on the same structural insights about how strings overlap.
Full proof set Master
Proposition 1 (KMP failure function correctness). For each , equals the length of the longest proper prefix of that is also a suffix of .
Proof. The failure function computation iterates from 2 to . At each step, it maintains the invariant that is the length of the longest proper prefix of that is also a suffix. If , then is a prefix-suffix of . Otherwise, it follows the chain of decreasing values (via ) until either a match is found or . By induction on , the invariant holds and is computed correctly.
Proposition 2 (KMP matching finds all occurrences). If occurs in at shift , then KMP reports shift .
Proof. When KMP processes text position , the pattern pointer equals (all characters matched). At this point, the algorithm reports the match at shift . The failure function ensures that never skips a valid partial match: if matches and the next character mismatches, then is set to , which is the longest proper prefix of that could start a new match. This is exactly the right amount of backtracking — no valid shift is missed.
Proposition 3 (suffix array search correctness). Binary search on the suffix array finds all occurrences of in .
Proof. The suffixes in SA order are lexicographically sorted. The pattern defines a lexicographic range: all suffixes starting with are contiguous in the sorted order. Binary search finds the leftmost and rightmost suffixes in this range. Each such suffix corresponds to an occurrence of at position in . Since all suffixes starting with are found, all occurrences are found.
Connections Master
47.01.01— DFAs are the foundation for the KMP and Aho-Corasick automata; each string-matching algorithm builds a specific DFA that recognises the pattern language.47.01.02— The Aho-Corasick automaton uses failure links analogous to the NFA subset construction: both convert nondeterministic state tracking into deterministic transitions.47.01.03— Regular expressions generalise exact string matching; the Thompson construction builds NFAs from regex that are processed by techniques similar to KMP and Aho-Corasick.47.04.01— Amortized analysis justifies the bound of KMP: the potential (pattern position) bounds total work across the entire matching phase.
Historical & philosophical context Master
The Knuth-Morris-Pratt algorithm was discovered independently by Knuth and Pratt (1970) and by Morris (1970), and published jointly in 1977. Knuth credits the key insight to Morris, who was implementing a text editor and needed an efficient search algorithm. The failure function was independently discovered by several researchers, highlighting that the prefix-suffix structure is a natural and unavoidable concept in string matching [CLRS §31.3 notes].
The Aho-Corasick algorithm was published by Alfred Aho and Margaret Corasick in 1975, originally developed for bibliographic search in the Bell Labs library system. The algorithm's elegance lies in converting the multi-pattern problem into an automaton construction, bridging string matching and automata theory. The failure links are the exact multi-pattern generalisation of the KMP failure function [Gusfield Ch. 2].
Suffix arrays were introduced by Manber and Myers in 1993 as a space-efficient alternative to suffix trees. While suffix trees use space with a constant factor of about 20 bytes per character, suffix arrays use only 4 bytes per character (one integer). The trade-off is slightly slower query times ( vs for suffix trees), but the space savings made suffix arrays practical for indexing large text corpora like genome sequences. The linear-time SA-IS construction algorithm (2011) closed the theoretical gap between suffix arrays and suffix trees [Gusfield Ch. 3].
Bibliography Master
@book{clrs2022,
author = {Cormen, Thomas H. and Leiserson, Ronald L. and Rivest, Clifford and Stein, Christopher},
title = {Introduction to Algorithms},
edition = {4th},
publisher = {MIT Press},
year = {2022},
}
@book{gusfield1997,
author = {Gusfield, Dan},
title = {Algorithms on Strings, Trees, and Sequences},
publisher = {Cambridge University Press},
year = {1997},
}
@article{aho1975efficient,
author = {Aho, Alfred V. and Corasick, Margaret J.},
title = {Efficient String Matching: An Aid to Bibliographic Search},
journal = {Communications of the ACM},
volume = {18},
number = {6},
pages = {333--340},
year = {1975},
}
@article{knuth1977fast,
author = {Knuth, Donald E. and Morris, James H. and Pratt, Vaughan R.},
title = {Fast Pattern Matching in Strings},
journal = {SIAM Journal on Computing},
volume = {6},
number = {2},
pages = {323--350},
year = {1977},
}