47.05.04 · theoretical-cs / 05-randomized-algorithms

FPRAS for Counting Problems: DNF Counting and the Permanent

shipped3 tiersLean: none

Anchor (Master): Arora and Barak 2009 Computational Complexity: A Modern Approach (Cambridge) Ch. 17; Jerrum, Sinclair and Vigoda 2004 A Polynomial-Time Approximation Algorithm for the Permanent of a Matrix with Nonnegative Entries (JACM) (simulated annealing on the switch-or-stay Markov chain)

Intuition Beginner

Counting is harder than deciding. Asking "does a satisfying assignment exist?" is one thing. Asking "how many satisfying assignments are there?" is another entirely. The first is NP. The second is #P, a class that captures counting problems, and it is believed to be much harder.

But we can often estimate the count without computing it exactly. Think of estimating the number of fish in a lake. You cannot count every fish, but you can catch some, tag them, release them, catch more, and see what fraction are tagged. This Monte Carlo approach gives an estimate whose accuracy improves with more samples.

A Fully Polynomial-Time Randomised Approximation Scheme (FPRAS) does exactly this for counting problems. Given a counting function and an error parameter , it runs in time polynomial in the input size and , and outputs an estimate within a factor of of the true count with high probability. The key insight is that random sampling, done carefully, can turn an intractable exact counting problem into a tractable approximate one.

Two landmark examples drive the theory. Counting satisfying assignments of a DNF formula has a simple FPRAS using the Karp-Luby method. Approximating the permanent of a matrix (equivalent to counting perfect matchings) has a sophisticated FPRAS via Markov chain Monte Carlo.

Visual Beginner

Exact counting vs. approximate counting:

  Formula: (x1 AND x2) OR (x3 AND NOT x4)
  
  Exact: enumerate all 16 assignments, count satisfying ones
    0000: 0    0001: 0    0010: 1    0011: 1
    0100: 0    0101: 0    0110: 1    0111: 1
    1000: 0    1001: 0    1010: 1    1011: 1
    1100: 1    1101: 1    1110: 1    1111: 1
  Count = 9

  FPRAS: sample random assignments, count fraction satisfying
    Sample 100 assignments -> ~56% satisfying -> estimate ~9.0
    (With enough samples, the estimate is within (1+eps) of 9)
Problem Exact counting Approximate (FPRAS?)
DNF satisfying assignments #P-complete Yes (Karp-Luby)
Perfect matchings (permanent) #P-complete Yes (JSV Markov chain)
SAT (CNF) satisfying assignments #P-complete No (unless NP = RP)
Independent sets in bipartite graphs #P-complete Yes (Markov chain)

Worked example Beginner

We estimate the number of satisfying assignments of a DNF formula using the Karp-Luby method.

Formula: over 4 variables.

Step 1: Identify the clauses. has satisfying assignments (the two specified variables are fixed, the other two are free). also has satisfying assignments.

Step 2: Compute the union size. The total number of assignments satisfying at least one clause is at most , but this overcounts assignments satisfying both clauses. Both and are satisfied when — only one assignment. So .

Step 3: Estimate via sampling. The Karp-Luby trick: instead of sampling from , sample uniformly from (the satisfying assignments) and use the estimator . The simpler version: sample a random clause with probability , then sample a random assignment from , and check if it satisfies any earlier clause.

The estimate is . With enough samples, this converges to 7.

Check your understanding Beginner

Formal definition Intermediate+

Definition (FPRAS). A fully polynomial-time randomised approximation scheme for a function is a randomised algorithm that, on input and , runs in time and outputs satisfying:

Definition (#P). A function is in #P if there exists a polynomial-time nondeterministic Turing machine such that .

Definition (permanent). The permanent of an matrix is:

Unlike the determinant, the permanent does not include signs from the permutation parity.

Definition (DNF counting). Given a DNF formula over variables, compute .

Counterexamples to common slips

  • "FPRAS means the algorithm is always correct." No. It has a failure probability that can be made arbitrarily small by repetition. A single run has at least 2/3 success probability.

  • "The permanent equals the determinant." The determinant has signs: . The permanent drops the signs. This small difference makes the determinant computable in polynomial time but the permanent #P-complete.

  • "If a problem has an FPRAS, then exact counting is also easy." No. DNF counting is #P-complete (so exact counting is hard), but it has an FPRAS. Approximate counting can be much easier than exact counting.

Key theorem with proof Intermediate+

Theorem (Karp-Luby FPRAS for DNF counting). There is an FPRAS for counting the number of satisfying assignments of a DNF formula.

Proof sketch. Let be a DNF formula over variables. For each clause , let be the set of assignments satisfying . We want .

The estimator. Define with . Each element of is a (clause, satisfying assignment) pair. Sample uniformly from by first picking clause with probability , then picking uniformly at random.

Define if the sampled pair satisfies for all (i.e., does not satisfy any earlier clause), and otherwise.

Expectation. . More precisely, each assignment contributes to exactly once (when it is sampled with its minimum-index satisfied clause). So .

Variance bound. The ratio is bounded by a polynomial because each clause has at most satisfying assignments, and the number of overlapping clauses is bounded by the formula structure. Taking independent samples and averaging gives a estimate with constant probability. Repeating and taking the median boosts to probability .

Bridge. The Karp-Luby estimator is the foundational reason DNF counting admits an FPRAS: by sampling from the union of satisfying sets via a weighted clause distribution, the variance is controlled enough for polynomial-sample convergence, and this builds toward the Markov chain methods needed for harder counting problems like the permanent where direct sampling from the solution space is impossible. This is exactly the pattern that appears again in the JSV algorithm where a rapidly-mixing Markov chain replaces uniform sampling, and it generalises to the entire field of approximate counting via self-reducibility. The central insight is that the gap between #P-completeness and approximability is bridged by random sampling with controlled variance, and putting these together with the hashing techniques of 47.05.01 yields the full machinery of approximate counting.

Exercises Intermediate+

Lean formalization Intermediate+

Mathlib has no formalization of FPRAS, the Karp-Luby estimator, or Markov chain Monte Carlo methods for counting. The permanent is defined in Mathlib (Matrix.perm), but there is no connection to approximate counting, no rapidly-mixing Markov chain formalization, and no self-reducibility telescoping product construction. A formalization of the Karp-Luby DNF counting FPRAS would be the natural entry point, requiring a FPRAS typeclass, a proof that the Karp-Luby estimator is unbiased, and a variance bound. This unit ships without a lean_module.

Advanced results Master

Three directions extend the basic FPRAS framework.

Result 1 (rapid mixing via canonical paths). The JSV Markov chain achieves rapid mixing in time . The proof uses the canonical paths method: for each pair of states , define a canonical path through the state space. The congestion of the canonical paths bounds the spectral gap, which bounds the mixing time. The key technical ingredient is the fact that the space of perfect and near-perfect matchings forms a connected graph under the switch operation, and the number of canonical paths through any transition edge is bounded by a polynomial in [JSV 2004].

Result 2 (FPRAS for other #P-complete problems). Several other #P-complete counting problems admit FPRAS: the number of independent sets in a bipartite graph (via the hard-core model Markov chain), the number of linear extensions of a partial order (via Karzanov-Khachiyan chain), and the partition function of the ferromagnetic Ising model (via Jerrum-Sinclair). The common technique is: identify a Markov chain on the solution space, prove rapid mixing (via canonical paths or coupling), and use the mixing time to bound the sample complexity for the estimator [Arora-Barak Ch. 17].

Result 3 (connections to approximate counting and approximate sampling). The equivalence between approximate counting and approximate sampling (Jerrum, Valiant and Vazirani 1986) states that for self-reducible problems, an FPRAS exists iff there is a polynomial-time algorithm for approximate uniform sampling from the solution space. This equivalence is the structural reason why FPRAS constructions always go through a sampling algorithm: the JSV chain samples matchings approximately uniformly, and the telescoping product converts this into a count estimate [Arora-Barak Ch. 17].

Synthesis. The FPRAS framework is the foundational reason that #P-completeness does not preclude efficient approximate counting: self-reducibility connects counting to sampling, and rapidly-mixing Markov chains provide the sampling engine. The central insight is that the variance of a well-designed estimator can be bounded independently of the exact count, so a polynomial number of samples suffices for a estimate. This builds toward the derandomisation questions of 47.05.05 where eliminating the randomness from these schemes would imply collapse of the polynomial hierarchy, and this is exactly the pattern that appears again in 47.02.07 where Toda's theorem shows that the counting hierarchy contains the entire polynomial hierarchy. Putting these together with the hashing techniques of 47.05.01 yields the full picture: randomised approximate counting is possible for a rich class of #P-complete problems, and the boundary between approximable and inapproximable counting problems maps to the boundary between NP and harder complexity classes.

Full proof set Master

Proposition 1 (Karp-Luby estimator is unbiased). The Karp-Luby estimator for satisfies .

Proof. The sample space is with uniform distribution. Define if for all , and otherwise.

For each , let . The pair satisfies the condition for , so .

For any other pair with and , we have with , so .

For pairs with and , this is impossible since .

So is nonzero only on pairs , one per element . Therefore:

Proposition 2 (sample complexity of FPRAS). If an unbiased estimator for satisfies for some constant , then independent copies of suffice to obtain a -estimate of with probability at least 2/3.

Proof. Let where the are independent copies of . By Chebyshev's inequality:

Setting makes this at most . So .

Connections Master

  • 47.05.01 — Universal and perfect hashing provide the randomness infrastructure used in counting: the Karp-Luby estimator uses random sampling, and the method of conditional expectations can derandomise parts of the scheme.

  • 47.02.07 — #P-completeness of the permanent (Valiant's theorem) establishes that exact counting is #P-hard; this unit shows approximate counting avoids the barrier.

  • 47.03.06 — Hardness of approximation for NP optimisation problems parallels the inapproximability results for #P counting problems (e.g., #SAT has no FPRAS unless NP = RP).

  • 47.05.05 — Derandomisation of FPRAS algorithms would give deterministic approximation schemes; the impossibility of doing so for some problems connects to the derandomisation landscape.

Historical & philosophical context Master

The theory of approximate counting was pioneered by Karp and Luby (1983) who gave the first FPRAS for DNF counting. The broader framework connecting counting and sampling was developed by Jerrum, Valiant and Vazirani (1986), who proved the equivalence of approximate counting and approximate sampling for self-reducible problems. The crowning achievement is the Jerrum-Sinclair-Vigoda algorithm (2004) for the permanent, which resolved a major open problem by providing an FPRAS for one of the most natural #P-complete functions.

The permanent has a storied history: it was defined by Cauchy in 1812 alongside the determinant, and its computational complexity was studied by Marcus and Minc from the 1960s. Valiant's 1979 proof that computing the permanent of a 0-1 matrix is #P-complete (even for 0-1 matrices) established that exact computation is intractable. The JSV algorithm showed that approximate computation is feasible, completing the picture [JSV 2004].

The philosophical lesson is that the gap between exact and approximate computation can be vast: a problem can be #P-complete (and hence at least as hard as any NP problem) yet admit an efficient randomised approximation. Randomness, in this context, is not a convenience but a necessity—derandomising the JSV algorithm would imply a collapse of the polynomial hierarchy.

Bibliography Master

@book{arora2009computational,
  author    = {Arora, Sanjeev and Barak, Boaz},
  title     = {Computational Complexity: A Modern Approach},
  publisher = {Cambridge University Press},
  year      = {2009},
}
@article{jerrum2004polynomial,
  author  = {Jerrum, Mark and Sinclair, Alistair and Vigoda, Eric},
  title   = {A Polynomial-Time Approximation Algorithm for the Permanent of a Matrix with Nonnegative Entries},
  journal = {Journal of the ACM},
  volume  = {51},
  number  = {4},
  pages   = {671--697},
  year    = {2004},
}
@inproceedings{karpluby1983monte,
  author    = {Karp, Richard M. and Luby, Michael},
  title     = {Monte-Carlo Algorithms for Enumeration and Reliability Problems},
  booktitle = {Proceedings of the 24th IEEE Symposium on Foundations of Computer Science},
  year      = {1983},
  pages     = {56--64},
}
@article{jerrum1986self,
  author  = {Jerrum, Mark R. and Valiant, Leslie G. and Vazirani, Vijay V.},
  title   = {Random Generation of Combinatorial Structures from a Uniform Distribution},
  journal = {Theoretical Computer Science},
  volume  = {43},
  pages   = {169--188},
  year    = {1986},
}