The PCP Theorem: Probabilistically Checkable Proofs and Hardness of Approximation
Anchor (Master): Arora & Barak 2009 Computational Complexity: A Modern Approach (Cambridge) §11.1-11.5 (Dinur's proof via gap amplification, the original ALMSS proof via low-degree testing and composition, hardness of approximating MAX-3SAT, vertex cover, and clique)
Intuition Beginner
A mathematician submits a proof to a journal. The referee reads every line carefully. This takes a long time. But what if the referee could verify the proof by reading only a few randomly chosen lines?
This sounds impossible. A proof with a single error could be wrong, and the error might be on any line. If the referee reads only three lines out of a thousand, the error could easily be missed.
The PCP theorem says it is possible — for a specific kind of proof system. The trick is that the proof is specially encoded. Instead of the usual linear format, the proof is written so that any error is "spread out" over many locations. A single logical mistake in the argument causes inconsistencies throughout the encoded proof. So reading a few random locations has a good chance of catching any error.
Think of it as a holographic proof. In a hologram, every piece contains information about the whole image. Similarly, every small piece of a PCP proof contains partial information about the whole argument. Checking a few pieces suffices because a false proof has inconsistencies everywhere.
More precisely: for any NP problem, there is a way to write proofs such that a verifier using logarithmically many coin flips and reading only a constant number of bits (say, 3) can distinguish correct proofs from incorrect ones with high probability. This constant number of queries does not depend on the input size.
The PCP theorem has a dramatic consequence for approximation. It implies that for many optimisation problems, finding even an approximate solution is as hard as finding the exact solution. For example, there is no polynomial-time algorithm that approximates the maximum number of satisfiable clauses in a 3SAT formula within a factor of (for any ) unless P = NP.
Visual Beginner
The PCP verification process:
Proof π (long string of bits)
┌──────────────────────────────────────────┐
│ 0 1 1 0 0 1 1 0 1 0 0 1 1 0 1 0 ... │
└──────────────────────────────────────────┘
↑ ↑ ↑
│ │ │
Verifier V reads only 3 bits (using random coins to choose which)
Coins: r = 0110... → positions 47, 203, 891
Check: does the constraint on those 3 bits hold?| Property | Description |
|---|---|
| Randomness | coin flips (polynomially many possible random strings) |
| Queries | bits of the proof (constant, independent of input size) |
| Completeness | If , there exists accepted with probability 1 |
| Soundness | If , every is rejected with probability |
The gap view of hardness of approximation:
MAX-3SAT optimum fraction of satisfiable clauses:
Yes-instance: ≥ 7/8 + ε of clauses can be satisfied
No-instance: ≤ 7/8 of clauses can be satisfied
Gap of ε between yes and no instances
→ Approximating within 7/8 + ε is NP-hardWorked example Beginner
Consider the connection between the PCP theorem and MAX-3SAT hardness.
MAX-3SAT asks: given a 3CNF formula, what fraction of clauses can be simultaneously satisfied? A random assignment satisfies of clauses in expectation (each clause has 3 literals, fails with probability ).
The PCP theorem implies that there exists such that distinguishing between "all clauses satisfiable" and "at most clauses satisfiable" is NP-hard.
To see why, start with a 3SAT instance . By the PCP theorem, there is an encoding that creates a new 3SAT-like instance where:
- If is satisfiable, then has an assignment satisfying fraction of clauses.
- If is unsatisfiable, then every assignment satisfies fraction of clauses.
If there were a polynomial-time algorithm approximating MAX-3SAT within , it could distinguish these two cases and hence decide 3SAT. This would imply P = NP.
This is the "gap" version of the PCP theorem: it creates a gap between the best achievable in yes-instances and no-instances, and this gap makes approximation NP-hard.
Check your understanding Beginner
Formal definition Intermediate+
Definition (PCP class). is the class of languages for which there exists a polynomial-time probabilistic verifier such that:
- On input of length , uses at most random bits.
- queries at most bits of the proof .
- Completeness: If , there exists such that .
- Soundness: If , then for every , .
Theorem (PCP theorem). .
The inclusion is the easier direction: the verifier uses random bits, so there are only polynomially many possible random strings. An NP verifier can simulate all of them deterministically and accept iff they all accept.
The inclusion is the deep direction. Two proofs exist:
Original proof (ALMSS 1998). Encode the NP computation as a low-degree polynomial (arithmetisation), and use self-testing and self-correction techniques from program checking to verify the polynomial with few queries. The proof uses algebraic techniques from coding theory and the theory of low-degree extensions.
Dinur's proof (2007). Start with a constraint graph (a graph where edges represent constraints on vertex labels) and iteratively amplify the gap between satisfiable and unsatisfiable instances by graph powering and composition. Each step increases the fraction of violated constraints in the no-case while preserving satisfiability in the yes-case.
Theorem (equivalent gap formulation). The PCP theorem is equivalent to: there exists such that given a 3CNF formula , it is NP-hard to distinguish:
- (Yes) is satisfiable.
- (No) Every assignment satisfies fraction of clauses.
Proof of equivalence. PCP gap: the PCP verifier for SAT produces a constraint system on variables per constraint, which can be converted to a 3CNF formula with a gap. Gap PCP: a gap instance provides the separation needed for the PCP soundness condition; the proof consists of the assignment, and the verifier checks random constraints.
Counterexamples to common slips
"The PCP theorem proves that NP proofs can be checked in constant time." The verifier runs in polynomial time (to compute which bits to query based on the randomness). Only the number of proof bits read is constant. The total running time is polynomial.
"Soundness 1/2 is the best possible." The soundness can be reduced to any constant less than 1 by repeating the PCP verification times (using additional random bits but still total).
"The PCP theorem only applies to SAT." The PCP theorem applies to every NP language. SAT is used as the starting point because it is NP-complete, but the theorem is a characterisation of the entire class NP.
Key theorem with proof Intermediate+
Theorem (NP PCP[, 1]). Every NP language has a probabilistically checkable proof with randomness and queries.
Proof sketch (Dinur's approach).
Step 1 (constraint graph formulation). An NP computation can be encoded as a constraint graph where each vertex has a label and each edge carries a constraint on . SAT maps to: is there a labelling satisfying all constraints?
Step 2 (gap creation). Define as the minimum fraction of unsatisfied edges over all labellings. If is satisfiable, . If is unsatisfiable, (but possibly very small).
Step 3 (gap amplification by powering). The graph power replaces each vertex by a neighbourhood of radius , and each edge of carries a constraint on the labels of all vertices in the two neighbourhoods. If , then where as . This amplifies a small gap to a constant gap.
Step 4 (alphabet reduction). The powering step increases the alphabet size. A composition step (using an "inner" PCP as a gadget) reduces the alphabet back to a constant without losing the gap.
Step 5 (iteration). Repeat gap amplification and alphabet reduction times until for a constant in the no-case.
Step 6 (PCP construction). The final constraint graph with constant gap gives a PCP: the proof is the vertex labelling, the verifier picks a random edge, reads the labels of its two endpoints ( bits), and checks the constraint. If the graph is satisfiable, all edges pass. If not, a random edge fails with probability .
Bridge. The PCP theorem builds toward the entire theory of hardness of approximation as the natural quantitative refinement of NP-completeness: the central insight is that NP-hardness does not merely prevent finding exact optima but also prevents finding good approximations for many problems, and this is exactly what makes the PCP theorem the most powerful tool in the theory of approximation algorithms. This appears again in 47.02.03 where the NP-completeness of 3SAT, Clique, and Vertex Cover is the starting point, and in 47.03.03 where the random-restriction techniques used in circuit lower bounds share a common probabilistic methodology with PCP constructions. The foundational reason the PCP theorem works is that algebraic encoding (low-degree polynomials) and combinatorial amplification (gap amplification) provide complementary methods for spreading information across a proof, and putting these together with the gap formulation shows that approximation hardness is not an isolated phenomenon but a structural property of NP-complete optimisation problems.
Exercises Intermediate+
Lean formalization Intermediate+
Mathlib has no formalisation of the PCP theorem, probabilistically checkable proofs, or hardness-of-approximation results. The foundational gap is the absence of a Complexity.PCP class definition, a formalised gap-amplification argument, and any hardness-of-approximation result for specific problems. The PCP theorem's proof is one of the longest and most technical in complexity theory, making it a significant formalisation challenge. This unit ships without a lean_module.
Advanced results Master
Three frontiers extend the PCP theorem.
Theorem 1 (Håstad's optimal 3-bit PCP). For any , , where the subscripts indicate completeness 1 and soundness . The verifier reads exactly 3 bits of the proof and accepts iff they satisfy a linear equation mod 2 [Arora & Barak §11.5].
This is optimal: 3 bits is the minimum needed for a meaningful PCP, and is the threshold for MAX-3SAT (since a random assignment satisfies of clauses).
Theorem 2 (unique games conjecture). Khot's Unique Games Conjecture (2002) states that a specific constraint satisfaction problem (where each constraint is a bijection on the labels) is NP-hard to approximate. If true, it gives tight hardness-of-approximation results for many problems: VERTEX-COVER cannot be approximated within , MAX-CUT cannot be approximated within , etc.
The UGC is a strengthened version of the PCP theorem that, if true, would resolve the approximability of many fundamental optimisation problems.
Theorem 3 (2-to-1 theorem). Dinur, Khot, Kindler, Minzer, and Safra (2018) proved that a variant of the label-cover problem with 2-to-1 constraints is NP-hard. This gives hardness of approximation results intermediate between the PCP theorem and UGC, including the result that approximating vertex cover within is NP-hard.
Synthesis. The PCP theorem is the foundational reason that NP-completeness has quantitative consequences for approximation: the central insight is that the gap between completeness and soundness in a PCP directly translates to a hardness-of-approximation result for an optimisation problem, and this is exactly what makes the PCP theorem the bridge between computational complexity and combinatorial optimisation. This builds toward the unique games conjecture which promises to resolve the approximability of virtually every fundamental NP-hard problem, and appears again in 47.02.03 where the NP-complete problems serve as the starting point for gap-producing reductions. The generalises from the qualitative statement "this problem is hard" to the quantitative statement "this problem is hard to approximate within factor ," and putting these together with Håstad's optimal result shows that the PCP framework gives matching upper and lower bounds for the approximability of MAX-3SAT.
Full proof set Master
Proposition 1 (PCP[, 1] NP). Every language in is in NP.
Proof. Let with verifier . The certificate is the proof string . The NP verifier enumerates all random strings . For each , it simulates on the same proof , reading bits. It accepts iff accepts for all .
If : by completeness, there exists accepted on all . NP verifier accepts.
If : by soundness, for every , rejects on of the . The NP verifier checks all and rejects.
Proposition 2 (gap-to-hardness reduction). If it is NP-hard to distinguish 3CNF formulas where all clauses are satisfiable from those where are satisfiable, then approximating MAX-3SAT within is NP-hard.
Proof. Let be a 3CNF formula. Use the gap-producing reduction to create . If is satisfiable, is fully satisfiable (value 1). If is unsatisfiable, has value . A -approximation for MAX-3SAT would return a value in the first case and in the second, distinguishing them. This decides SAT in polynomial time, so P = NP.
Connections Master
47.02.02— The Cook-Levin theorem and NP-completeness of SAT provide the starting point for the PCP construction.47.02.03— Classic NP-complete problems (MAX-3SAT, Clique, Vertex Cover) are the targets of hardness-of-approximation results derived from the PCP theorem.47.03.03— The random-restriction techniques used in circuit lower bounds share probabilistic methodology with PCP constructions (randomness for amplification).47.03.04— Interactive proofs (IP = PSPACE) and PCP both use randomised verification; IP extends to PSPACE while PCP characterises NP.47.04.04— Bipartite matching (in P) contrasts with the NP-hard approximability of problems like vertex cover on general graphs.
Historical & philosophical context Master
The PCP theorem was proved by Sanjeev Arora, Carsten Lund, Rajeev Motwani, Madhu Sudan, and Mario Szegedy in 1992 (published in JACM 1998). Their proof used algebraic techniques: arithmetising NP computations as low-degree polynomials and using self-testing/correcting methods from the theory of program checking. This work earned them the Gödel Prize in 2001 [Arora & Barak §11.2].
Irit Dinur's 2007 proof was a major simplification. She replaced the algebraic machinery with a purely combinatorial gap-amplification argument, making the PCP theorem accessible to a wider audience. Her proof iteratively amplifies the unsatisfiability gap of a constraint graph by graph powering and composition, and was recognised with the Michael O. Rabin Prize.
The PCP theorem resolved a question posed by Feige, Goldwasser, Lovász, Safra, and Szegedy (1991) who first showed the connection between interactive proof systems and hardness of approximation. The philosophical significance is that the difficulty of NP-complete problems extends beyond finding exact solutions: even finding good approximations is intractable, and the quality of approximation that is achievable is precisely quantified by the PCP's soundness parameter.
Johan Håstad's 2001 result gave the optimal 3-bit PCP with soundness , which is the strongest possible for any 3-query PCP testing linear equations. This matched the simple random-assignment algorithm that achieves for MAX-3SAT, giving a tight result.
Bibliography Master
@book{arora2009computational,
author = {Arora, Sanjeev and Barak, Boaz},
title = {Computational Complexity: A Modern Approach},
publisher = {Cambridge University Press},
year = {2009},
}
@article{almss1998proof,
author = {Arora, Sanjeev and Lund, Carsten and Motwani, Rajeev and Sudan, Madhu and Szegedy, Mario},
title = {Proof Verification and the Hardness of Approximation Problems},
journal = {Journal of the ACM},
volume = {45},
number = {3},
pages = {501--555},
year = {1998},
}
@article{dinur2007pcp,
author = {Dinur, Irit},
title = {The {PCP} Theorem by Gap Amplification},
journal = {Journal of the ACM},
volume = {54 third},
number = {2},
pages = {1--44},
year = {2007},
}
@article{hastad2001optimal,
author = {H{\aa}stad, Johan},
title = {Some Optimal Inapproximability Results},
journal = {Journal of the ACM},
volume = {48},
number = {4},
pages = {798--859},
year = {2001},
}
@article{khot2002unique,
author = {Khot, Subhash},
title = {On the Power of Unique 2-Prover 1-Round Games},
journal = {Proceedings of the 34th ACM Symposium on Theory of Computing},
pages = {767--775},
year = {2002},
}