The Proof (continued)

Cook’s Theorem: the First NP-Complete Problem

Theorem 39 (Cook, 1971) sat is NP-complete.

• sat ∈ NP (p. 120).

• circuit sat reduces to sat (p. 294).

• By Proposition 33 (p. 311), it remains to show that all languages in NP can be reduced to circuit sat.^a

aAs a bonus, this also shows circuit sat is NP-complete.

The Proof (continued)

• Let single-string NTM M decide L ∈ NP in time n^k.

• Assume M has exactly two nondeterministic choices at each step: choices 0 and 1.

• For each input x, we construct circuit R(x) such that x ∈ L if and only if R(x) is satisfiable.

• Equivalently, for each input x, M(x) = “yes” for some computation path if and only if R(x) is satisfiable.

• How to come up with a polynomial-sized R(x) when there are exponentially many computation paths?

The Proof (continued)

• A straightforward proof is to construct a variable-free circuit R_i(x) for the ith computation path.^a

• Then add a small circuit to output 1 if and only if there is an R_i(x) that outputs a “yes.”

• Clearly, the resulting circuit outputs 1 if and only if M accepts x.

• But, it is too large because there are exponentially many computation paths.

• Need to do better.

aThe circuit for Theorem 34 (p. 318) will do.

The Proof (continued)

• A sequence of nondeterministic choices is a bit string B = (c₁, c₂, . . . , c_{| x |}k−1) ∈ { 0, 1 }^{| x |k−1}.

• Once B is given, the computation is deterministic.

• Each choice of B results in a deterministic polynomial-time computation.

• Each circuit C at time i has an extra binary input c corresponding to the nondeterministic choice:

C(T_i−1,j−1, T_i−1,j, T_i−1,j+1, c) = T_ij.

The Proof (continued)

C

c

The Computation Tableau for NTMs and R(x)

# , - . / 0 1 

#

#





     

     

     

._ ._ ._

The Proof (concluded)

• Note that c1, c₂, . . . , c_{| x |}k−1 constitute the variables of R(x).

– Some call them the choice gates of the circuit.

• The overall circuit R(x) (on p. 338) is satisfiable if and only if there is a truth assignment B such that the

computation table accepts.

• This happens if and only if M accepts x, i.e., x ∈ L.

Stephen Arthur Cook

(1939–)

Richard Karp, “It is to our everlasting shame that we were unable to persuade the math department [of UC-Berkeley] to give him tenure.”

aTuring Award (1982). See http://conservancy.umn.edu/handle/107226 for an interview in 2002.

A Corollary

The construction in the above proof yields the following, more general result.

Corollary 40 If L ∈ NTIME(T (n)), then a

nondeterministic circuit with O(T²(n)) gates can decide L.

NP-Complete Problems

Wir m¨ussen wissen, wir werden wissen.

(We must know, we shall know.)

— David Hilbert (1900)

I predict that scientists will one day adopt a new principle: “NP-complete problems are hard.”

That is, solving those problems efficiently is impossible on any device that could be built in the real world, whatever the final laws of physics turn out to be.

— Scott Aaronson (2008)

Two Notions

• Let R ⊆ Σ^∗ × Σ^∗ be a binary relation on strings.

• R is called polynomially decidable if { x; y : (x, y) ∈ R } is in P.

• R is said to be polynomially balanced if (x, y) ∈ R implies | y | ≤ | x |^k for some k ≥ 1.^a

aIf the polynomial bound is replaced by a different class of func- tions, different complexity classes may arise. Contributed by Mr. Vincent Hwang (R10922138) on November 11, 2021.

An Alternative Characterization of NP

Proposition 41 (Edmonds, 1965) Let L ⊆ Σ^∗ be a

language. Then L ∈ NP if and only if there is a polynomially decidable and polynomially balanced relation R such that

L = { x : ∃y (x, y) ∈ R }.

• Suppose such an R exists.

• L can be decided by this NTM:

– On input x, the NTM guesses a y of length ≤ | x |^k. – It then tests if (x, y) ∈ R in polynomial time.

– It returns “yes” if the test is positive.

The Proof (concluded)

• Now suppose L ∈ NP.

• NTM N decides L in time | x |^k.

• Define R as follows: (x, y) ∈ R if and only if y is the encoding of an accepting computation of N on input x.

• R is polynomially balanced as N runs in polynomial time.

• R is polynomially decidable because it can be efficiently verified by consulting N ’s transition function.

• Finally L = { x : (x, y) ∈ R for some y } because N decides L.

Jack Edmonds (1934–)

Comments

• Any “yes” instance x of an NP problem has at least one succinct certificate or polynomial witness y.

• “No” instances have none.

• Certificates are short and easy to verify.

– An alleged satisfying truth assignment for sat; an alleged Hamiltonian path for hamiltonian path.

• Certificates may be hard to generate,^a but verification must be easy.

• NP is thus the class of easy-to-verify^b problems.

aUnless P equals NP.

bThat is, in polynomial time.

Comments (concluded)

• The degree k is not an input.

• How to find the k needed by the NTM is of no concern.^a

• We only need to prove there exists an NTM that accepts L in nondeterministic polynomial time.

• If R is not required to be polynomial balanced, then its running time may not be polynomial in | x |.^b

– It is still polynomial in | x | + | y |.

aContributed by Mr. Kai-Yuan Hou (B99201038, R03922014) on November 3, 2015.

bThanks to a lively discussion on November 11, 2021.

You Have an NP-Complete Problem (for Your Thesis)

• From Propositions 29 (p. 307) and 32 (p. 310), it is the least likely to be in P.

• Your options are:

– Approximations.

– Special cases.

– Average performance.

– Randomized algorithms.

– Exponential-time algorithms that work well in practice.

– “Heuristics” (and pray that it works for your thesis).

I thought NP-completeness was an interesting idea:

I didn’t quite realize its potential impact.

— Stephen Cook (1998)

I was indeed surprised by Karp’s work since I did not expect so many wonderful problems were NP-complete.

— Leonid Levin (1998)

Correct Use of Reduction in Proving NP-Completeness

• Recall that L1 reduces to L₂ if there is an efficient function R such that for all inputs x (p. 279),

x ∈ L₁ if and only if R(x) ∈ L₂.

• When L₁ is known to be NP-complete and when L₂ ∈ NP, then L₂ is NP-complete.^a

• A common mistake is to focus on solving x ∈ L₁ or solving R(x) ∈ L2.

• The correct way is to, given a certificate for x ∈ L1 (a satisfying truth assignment, e.g.), construct a certificate for R(x) ∈ L₂ (a Hamiltonian path, e.g.), and vice versa.

aProposition 33 (p. 311).

3sat

• k-sat, where k ∈ Z⁺, is the special case of sat.

• The formula is in CNF and all clauses have exactly k literals (repetition of literals is allowed).

• For example,

(x₁ ∨ x2 ∨ ¬x3) ∧ (x1 ∨ x1 ∨ ¬x2) ∧ (x1 ∨ ¬x2 ∨ ¬x3).

3sat Is NP-Complete

• Recall Cook’s Theorem (p. 333) and the reduction of circuit sat to sat (p. 294).

• The resulting CNF has at most 3 literals for each clause.

– This accidentally shows that 3sat where each clause has at most 3 literals is NP-complete.

• Finally, duplicate one literal once or twice to make it a 3sat formula.

– So

x₁ ∨ x2 becomes x₁ ∨ x2 ∨ x2.

aGarey, Johnson, & Stockmeyer (1976).

Michael R. Garey (1945–)

David S. Johnson (1945–)

Larry Stockmeyer (1948–2004)

The Satisfiability of Random 3sat Expressions

• Consider a random 3sat expression φ with n variables and cn clauses.

• Each clause is chosen independently and uniformly from the set of all possible clauses.

• Intuitively, the larger the c, the less likely φ is satisfiable as more constraints are added.

• Indeed, there is a cn such that for c < c_n(1 − ), φ is satisfiable almost surely, and for c > c_n(1 + ), φ is unsatisfiable almost surely.^a

aFriedgut & Bourgain (1999). As of 2006, 3.52 < c_n < 4.596.

Another Variant of 3sat

Proposition 42 3sat is NP-complete for expressions in which each variable is restricted to appear at most three times, and each literal at most twice. (3sat here requires only that each clause has at most 3 literals.)

The Proof (continued)

• Consider a general 3sat expression in which x appears k times.

• Replace the first occurrence of x by x₁, the second by x₂, and so on.

– x₁, x₂, . . . , x_k are k new variables.

The Proof (concluded)

• Add (¬x1 ∨ x2) ∧ (¬x2 ∨ x3) ∧ · · · ∧ (¬xk ∨ x1) to the expression.

– It is logically equivalent to

x₁ ⇒ x2 ⇒ · · · ⇒ xk ⇒ x1.

– So it is necessary that x₁, x₂, . . . , x_k assume an

identical truth value to satisfy the whole expression.

• Note that each clause ¬x_i ∨ x_j above has only 2 literals.

• The resulting equivalent expression satisfies the conditions for x.

An Example

• Suppose we are given the following 3sat expression

· · · (¬x ∨ w ∨ g) ∧ · · · ∧ (x ∨ y ∨ z) · · · .

• The transformed expression is

· · · ( ¬x1 ∨w∨g)∧· · ·∧( x₂ ∨y∨z) · · · ( ¬x1 ∨ x₂ )∧( ¬x2 ∨ x₁ ).

– Variable x₁ appears 3 times.

– Literal x₁ appears once.

– Literal ¬x1 appears 2 times.

2sat Is in NL ⊆ P

• Let φ be an instance of 2sat: Each clause has 2 literals.

• NL is a subset of P.^a

• Because coNL = NL,^b we need to show only that recognizing unsatisfiable 2sat expressions is in NL.

• See the textbook for the complete proof.

aRecall p. 257.

bRecall p. 271.

Generalized 2sat: max2sat

• Consider a 2sat formula.

• Let K ∈ N.

• max2sat asks whether there is a truth assignment that satisfies at least K of the clauses.

– max2sat becomes 2sat when K equals the number of clauses.

Generalized 2sat: max2sat (concluded)

• max2sat can be used to solve the related optimization problem.

– With binary search, one can nail the maximum number of satisfiable clauses of 2sat formulas.

• max2sat ∈ NP: Guess a truth assignment and verify the count.

• We now reduce 3sat to max2sat.

max2sat Is NP-Complete

• Consider the following 10 clauses:

(x) ∧ (y) ∧ (z) ∧ (w)

(¬x ∨ ¬y) ∧ (¬y ∨ ¬z) ∧ (¬z ∨ ¬x) (x ∨ ¬w) ∧ (y ∨ ¬w) ∧ (z ∨ ¬w)

• Let the 2sat formula r(x, y, z, w) represent the conjunction of these clauses.

• The clauses are symmetric with respect to x, y, and z.

• How many clauses can we satisfy?

aGarey, Johnson, & Stockmeyer (1976).

The Proof (continued)

All of x, y, z are true: By setting w to true, we satisfy 4 + 0 + 3 = 7 clauses, whereas by setting w to false, we satisfy only 3 + 0 + 3 = 6 clauses.

Two of x, y, z are true: By setting w to true, we satisfy 3 + 2 + 2 = 7 clauses, whereas by setting w to false, we satisfy 2 + 2 + 3 = 7 clauses.

The Proof (continued)

One of x, y, z is true: By setting w to false, we satisfy 1 + 3 + 3 = 7 clauses, whereas by setting w to true, we satisfy only 2 + 3 + 1 = 6 clauses.

None of x, y, z is true: By setting w to false, we satisfy 0 + 3 + 3 = 6 clauses, whereas by setting w to true, we satisfy only 1 + 3 + 0 = 4 clauses.

The Proof (continued)

• A truth assignment that satisfies x ∨ y ∨ z can be extended to satisfy 7 of the 10 clauses of r(x, y, z, w), and no more.

• A truth assignment that does not satisfy x ∨ y ∨ z can be extended to satisfy only 6 of them, and no more.

• The reduction from 3sat φ to 2sat R(φ):

– For each clause C_i = (α ∨ β ∨ γ) of φ, add r(α, β, γ, w_i) to R(φ).

• If φ has m clauses, then R(φ) has 10m clauses.

The Proof (continued)

• Finally, set K = 7m.

• So the reduction transforms φ to (R(φ), 7m).

• We now show that K clauses of R(φ) can be satisfied if and only if φ is satisfiable.

The Proof (continued)

• Suppose K = 7m clauses of R(φ) can be satisfied.

– 7 clauses of each r(α, β, γ, w_i) must be satisfied because it can have at most 7 clauses satisfied.^a

– Hence each clause C_i = (α ∨ β ∨ γ) of φ is satisfied by the same truth assignment.

– So φ is satisfied.

aIf 70% of the world population are male and if at most 70% of each country’s population are male, then each country must have exactly 70%

male population.

The Proof (concluded)

• Suppose φ is satisfiable.

– Let T satisfy all clauses of φ.

– Each r(α, β, γ, w_i) can set its w_i appropriately to have 7 clauses satisfied.

– So K = 7m clauses are satisfied.

naesat

• The naesat (for “not-all-equal” sat) is like 3sat.

• But there must be a satisfying truth assignment under which no clauses have all three literals equal in truth value.

– Equivalently, there is a truth assignment such that each clause has a literal assigned true and a literal assigned false.

– Equivalently, there is a satisfying truth assignment under which each clause has a literal assigned false.

naesat (concluded)

• Take

φ = (¬x₁ ∨ ¬x₂ ∨ ¬x₃) ∧ (¬x₁ ∨ x₂ ∨ ¬x₃)

∧ (x1 ∨ x2 ∨ x3) as an example.

• Then { x₁ = true, x₂ = false, x₃ = false } nae-satisfies φ because

(false ∨ true ∨ true) ∧ (false ∨ false ∨ true)

∧ (true ∨ false ∨ false).

naesat Is NP-Complete

• Recall the reduction of circuit sat to sat on p. 294ff.

• It produced a CNF φ in which each clause has 1, 2, or 3 literals.

• Add the same variable z to all clauses with fewer than 3 literals to make it a 3sat formula.

• Goal: The new formula φ(z) is nae-satisfiable if and only if the original circuit is satisfiable.

aSchaefer (1978).

The Proof (continued)

• The following simple observation will be useful.

• Suppose T nae-satisfies a boolean formula φ.

• Let ¯T take the opposite truth value of T on every variable.

• Then ¯T also nae-satisfies φ.^a

aHesse’s Siddhartha (1922), “The opposite of every truth is just as true!”

The Proof (continued)

• Suppose T nae-satisfies φ(z).

– ¯T also nae-satisfies φ(z).

– Under T or ¯T , variable z takes the value false.

– This truth assignment T must satisfy all the clauses of φ.

∗ Because z is not the reason that makes φ(z) true under T anyway.

– So T |= φ.

– And the original circuit is satisfiable.

The Proof (concluded)

• Suppose there is a truth assignment that satisfies the circuit.

– Then there is a truth assignment T that satisfies every clause of φ.

– Extend T by adding T (z) = false to obtain T^. – T^ satisfies φ(z).

– Clearly, in no clauses are all three literals false under T^.

– In no clauses are all three literals true under T^.

∗ Need to go over the detailed construction on pp. 295–297.

Undirected Graphs

• An undirected graph G = (V, E) has a finite set of nodes, V , and a set of undirected edges, E.

• It is like a directed graph except that the edges have no directions and there are no self-loops.

• Use [ i, j ] to mean there is an undirected edge between node i and node j.^a

aAn equally good notation is { i, j }.

Independent Sets

• Let G = (V, E) be an undirected graph.

• I ⊆ V .

• I is independent if there is no edge between any two nodes i, j ∈ I.

• independent set: Given an undirected graph and a goal K, is there an independent set of size K?

• Many applications.

independent set Is NP-Complete

• This problem is in NP: Guess a set of nodes and verify that it is independent and meets the count.

• We will reduce 3sat to independent set.

• Note: If a graph contains a triangle, any independent set can contain at most one node of the triangle.

• The reduction will output graphs whose nodes can be partitioned into disjoint triangles, one for each clause.^a

aRecall that a reduction does not have to be an onto function.

The Proof (continued)

• Let φ be a 3sat formula with m clauses.

• We will construct graph G with K = m.

• Furthermore, φ is satisfiable if and only if G has an independent set of size K.

• Here is the reduction:

– There is a triangle for each clause with the literals as the nodes’ labels.

– Add edges between x and ¬x for every variable x.

(x

∨ x

∨ x

) ∧ (¬x

∨ ¬x

∨ ¬x

) ∧ (¬x

∨ x

∨ x

)

» [_

» [_ » [_

[_

[_ [_

» [_

[_ [_

Same literal labels that appear in the same clause or different clauses yield distinct nodes.

The Proof (continued)

• Suppose G has an independent set I of size K = m.

– An independent set can contain at most m nodes, one from each triangle.

– So I contains exactly one node from each triangle.

– Truth assignment T assigns true to those literals in I.

– T is consistent because contradictory literals are connected by an edge; hence both cannot be in I.

– T satisfies φ because it has a node from every triangle, thus satisfying every clause.^a

aThe variables without a truth value can be assigned arbitrarily. Con-

The Proof (concluded)

• Suppose φ is satisfiable.

– Let truth assignment T satisfy φ.

– Collect one node from each triangle whose literal is true under T .

– The choice is arbitrary if there is more than one true literal.

– This set of m nodes must be independent by construction.

∗ Because both literals x and ¬x cannot be assigned true.

Other independent set-Related NP-Complete Problems

Corollary 43 independent set is NP-complete for 4-degree graphs.

Theorem 44 independent set is NP-complete for planar graphs.

Theorem 45 (Garey & Johnson, 1977)) independent set is NP-complete for 3-degree planar graphs.

Is independent edge set Also NP-Complete?

• independent edge set: Given an undirected graph and a goal K, is there an independent edge set of size K?

• This problem is equivalent to maximum matching!

• Maximum matching can be solved in polynomial time.^a

aEdmonds (1965); Micali & V. Vazirani (1980).

A Maximum Matching

node cover

• We are given an undirected graph G and a goal K.

• node cover: Is there a set C with K or fewer nodes such that each edge of G has at least one of its

endpoints (i.e., incident nodes) in C?

• Many applications.

node cover (concluded)

node cover Is NP-Complete

Corollary 46 (Karp, 1972) node cover is NP-complete.

• I is an independent set of G = (V, E) if and only if V − I is a node cover of G.^a

I

aFinish the reduction!

Richard Karp

(1935–)

aTuring Award (1985).

Remarks

• Are independent set and node cover in P if K is a constant?

– Yes, because one can do an exhaustive search on all the possible node covers or independent sets (both

_n

K

 = O(n^K) of them, a polynomial).^b

• Are independent set and node cover NP-complete if K is a linear function of n?

– independent set with K = n/3 and node cover with K = 2n/3 remain NP-complete by our

reductions.

aContributed by Mr. Ching-Hua Yu (D00921025) on October 30, 2012.

bn = | V |.

clique

• We are given an undirected graph G and a goal K.

• clique asks if there is a set C with K nodes such that there is an edge between any two nodes i, j ∈ C.

• Many applications.

clique (concluded)

clique Is NP-Complete

Corollary 47 clique is NP-complete.

• Let ¯G be the complement of G, where [ x, y ] ∈ ¯G if and only if [ x, y ] ∈ G.

• I is a clique in G ⇔ I is an independent set in ¯G.