home > computation > complexity
SeriesLimits of Computationpart 3 of 3

The three barriers standing in front of P vs NP

by Anil Kulkarni18 min readCOMPUTATION

Half a century in, P vs NP is not merely unsolved — we can prove theorems about why our proofs fail. Three times, complexity theorists have caught their own best techniques in the act of being insufficient. Those three results are the most detailed map of the problem anyone has drawn.

The question itself is stark: if a solution can be checked quickly, can it always be found quickly? Nearly everyone believes the answer is no. The trouble is not a shortage of effort but a surplus of self-knowledge — theorems showing that entire categories of argument are too coarse to decide the matter. Each barrier takes a technique that succeeded brilliantly elsewhere and proves, rigorously, that it cannot succeed here. The results have a strange flavour — mathematics auditing its own toolbox — and they repay being read in sequence, because each one answers the hope raised by the one before it.

Three proof techniques, each stopped by its own barrier short of the conjecture known techniques the goal diagonalization relativization BGS 1975 circuit counting natural proofs RR 1997 arithmetization algebrization AW 2008 P ≠ NP
FIG 1 Each technique advances until a barrier theorem stops it. Arithmetization travels farthest — it escaped relativization in 1992 — before meeting a wall of its own. No known technique reaches the box.

Three theorems about proofs

The first barrier arrived in 1975. Diagonalization — the workhorse that separated the computable from the uncomputable — treats programs as black boxes: it only ever runs them. Baker, Gill and Solovay noticed that any such argument survives unchanged if every machine is handed the same oracle, a free subroutine answering membership questions about some fixed set.1 Then they built an oracle world where P equals NP, and another where the two provably differ. A proof by pure #relativization would carry into both worlds and contradict one of them. Diagonalization alone can never decide the question.

So attention turned to circuits — concrete hardware, no black boxes. The 1980s produced genuine lower bounds against restricted circuit classes, and optimism ran high. Then @Razborov and Rudich looked at the successful arguments and noticed they shared two properties: each identified a feature of hard functions that was efficiently checkable and held for most functions. They called such proofs natural, and showed that any natural proof against general circuits would double as an efficient test distinguishing pseudorandom functions from truly random ones — breaking the cryptographic generators whose security rests on the very hardness beliefs that make P ≠ NP plausible.2 A proof in the style of #natural-proofs would refute its own working assumptions.

The third barrier answered a hope. Arithmetization — replacing Boolean formulas with polynomials over a field — powered the proof that IP equals PSPACE, a theorem that is false in some oracle worlds and true in ours, the first hard evidence that relativization could be escaped.3 In 2008, Aaronson and @Wigderson measured how far the escape reaches. Give the oracle an algebraic extension — let machines query a low-degree polynomial agreeing with it — and the old construction goes through again: worlds where the classes collapse, worlds where they split. Every technique whose queries respect that algebraic structure stops here. The barrier is called #algebrization, and arithmetization does not get past it.

Barriers are knowledge

It is tempting to read this as fifty years of failure. It is closer to the opposite. Each barrier is a theorem about the space of possible proofs — a negative-space portrait of the argument that must eventually work. We now know it must be non-relativizing, non-naturalizing, and non-algebrizing all at once, and that shortlist disqualifies nearly every tool in the standard kit.

That is why the surviving programs look so strange from the outside. Geometric complexity theory reaches for algebraic geometry and representation theory; proof-complexity approaches study the strength of the logical systems doing the proving; both were designed, quite explicitly, to slip past all three walls at once. Nobody knows which door opens, or whether a fourth barrier is waiting to be discovered behind the third. But the walls we have are mapped to the brick, and in mathematics a precisely located wall is not an obstacle report. It is a result.

Series complete You finished Limits of Computation — see all 3 parts

Notes

  1. An oracle is a black box bolted onto a machine: one step, one membership answer about a fixed set A. Diagonalization never inspects the internals of the machines it defeats — it only runs them — which is exactly why it survives the bolt-on, and exactly why it is too coarse to see the difference between P and NP [1].
  2. "Efficiently checkable" and "holds for most functions" are the technical conditions of constructivity and largeness. The generators broken would be those built from factoring or discrete logarithms — so a natural proof of P ≠ NP would demolish the strongest practical evidence that hardness exists at all [2].
  3. Shamir's 1992 theorem, building on the interactive-proof arithmetization of Lund, Fortnow, Karloff and Nisan. For about fifteen years it was the standard answer to "can we get past relativization?" — until algebrization showed the answer was "yes, but only this far" [3].

References

  1. Baker, T., Gill, J. & Solovay, R. (1975). "Relativizations of the P =? NP question." SIAM Journal on Computing 4(4), 431–442. doi:10.1137/0204037
  2. Razborov, A. A. & Rudich, S. (1997). "Natural proofs." Journal of Computer and System Sciences 55(1), 24–35. doi:10.1006/jcss.1997.1494
  3. Aaronson, S. & Wigderson, A. (2009). "Algebrization: a new barrier in complexity theory." ACM Transactions on Computation Theory 1(1), Article 2.