The Search Procedure

Maximum Satisfiability

• Given a set of clauses, maxsat seeks the truth assignment that satisfies the most simultaneously.

• max2sat is already NP-complete (p. 349), so maxsat is NP-complete.

• Consider the more general k-maxgsat for constant k.

– Let Φ = { φ₁, φ₂, . . . , φ_m } be a set of boolean expressions in n variables.

– Each φ_i is a general expression involving up to k variables.

– k-maxgsat seeks the truth assignment that satisfies the most expressions simultaneously.

A Probabilistic Interpretation of an Algorithm

• Let φ_i involve k_i ≤ k variables and be satisfied by s_i of the 2^ki truth assignments.

• A random truth assignment ∈ { 0, 1 }ⁿ satisfies φ_i with probability p(φ_i) = s_i/2^ki.

– p(φ_i) is easy to calculate as k is a constant.

• Hence a random truth assignment satisfies an average of p(Φ) =

m i=1

p(φ_i) expressions φ_i.

The Search Procedure

• Clearly

p(Φ) = p(Φ[ x₁ = true ]) + p(Φ[ x1 = false ])

2 .

• Select the t₁ ∈ { true, false } such that p(Φ[ x₁ = t₁ ]) is the larger one.

• Note that p(Φ[ x₁ = t₁ ]) ≥ p(Φ).

• Repeat the procedure with expression Φ[ x₁ = t₁ ] until all variables x_i have been given truth values t_i and all φ_i are either true or false.

The Search Procedure (continued)

• By our hill-climbing procedure, p(Φ)

≤ p(Φ[ x1 = t₁ ])

≤ p(Φ[ x1 = t₁, x₂ = t₂ ])

≤ · · ·

≤ p(Φ[ x₁ = t₁, x₂ = t₂, . . . , x_n = t_n ]).

• So at least p(Φ) expressions are satisfied by truth assignment (t₁, t₂, . . . , t_n).

The Search Procedure (concluded)

• Note that the algorithm is deterministic!

• It is called the method of conditional expectations.^a

aErd˝os & Selfridge (1973); Spencer (1987).

Approximation Analysis

• The optimum is at most the number of satisfiable φ_i—i.e., those with p(φ_i) > 0.

• The ratio of algorithm’s output vs. the optimum is^a

≥ p(Φ)



p(φi)>0 1 =



i p(φ_i)



p(φi)>0 1 ≥ min

p(φi)>0p(φ_i).

• This is a polynomial-time -approximation algorithm with  = 1 − min_p(φi)>0 p(φ_i) by Eq. (20) on p. 732.

• Because p(φ_i) ≥ 2^−k for a satisfiable φ_i, the heuristic is a polynomial-time -approximation algorithm with

 = 1 − 2^−k.

 

Back to maxsat

• In maxsat, the φi’s are clauses (like x ∨ y ∨ ¬z).

• Hence p(φ_i) ≥ 1/2 (why?).

• The heuristic becomes a polynomial-time

-approximation algorithm with  = 1/2.^a

• Suppose we set each boolean variable to true with probability (√

5 − 1)/2, the golden ratio.

• Then follow through the method of conditional expectations to derandomize it.

aJohnson (1974).

Back to maxsat (concluded)

• We will obtain a [ (3 − √

5 ) ]/2-approximation algorithm.^a

– Note [ (3 − √

5 ) ]/2 ≈ 0.382.

• If the clauses have k distinct literals, p(φ_i) = 1 − 2^−k.

• The heuristic becomes a polynomial-time

-approximation algorithm with  = 2^−k.

– This is the best possible for k ≥ 3 unless P = NP.

• All the results hold even if clauses are weighted.

max cut Revisited

• max cut seeks to partition the nodes of graph

G = (V, E) into (S, V − S) so that there are as many edges as possible between S and V − S.

• It is NP-complete (p. 384).

• Local search starts from a feasible solution and

performs “local” improvements until none are possible.

• Next we present a local-search algorithm for max cut.

A 0.5-Approximation Algorithm for max cut

1: S := ∅;

2: while ∃v ∈ V whose switching sides results in a larger cut do

3: Switch the side of v;

4: end while

5: return S;

Analysis

V₃ V₄

V₂ V₁

Optimal cut

Our cut

e₁₂

e₁₃

e₂₄

e₃₄ e₁₄ e₂₃

Analysis (continued)

• Partition V = V₁ ∪ V₂ ∪ V₃ ∪ V₄, where

– Our algorithm returns (V₁ ∪ V2, V₃ ∪ V4).

– The optimum cut is (V₁ ∪ V3, V₂ ∪ V4).

• Let eij be the number of edges between V_i and V_j.

• Our algorithm returns a cut of size

e₁₃ + e₁₄ + e₂₃ + e₂₄.

• The optimum cut size is

e₁₂ + e₃₄ + e₁₄ + e₂₃.

Analysis (continued)

• For each node v ∈ V₁, its edges to V₃ ∪ V₄ cannot be outnumbered by those to V₁ ∪ V₂.

– Otherwise, v would have been moved to V₃ ∪ V4 to improve the cut.

• Considering all nodes in V1 together, we have 2e₁₁ + e₁₂ ≤ e13 + e₁₄.

– 2e₁₁, because each edge in V₁ is counted twice.

• The above inequality implies

e₁₂ ≤ e13 + e₁₄.

Analysis (concluded)

• Similarly,

e₁₂ ≤ e23 + e₂₄ e₃₄ ≤ e₂₃ + e₁₃ e₃₄ ≤ e₁₄ + e₂₄

• Add all four inequalities, divide both sides by 2, and add the inequality e₁₄ + e₂₃ ≤ e₁₄ + e₂₃ + e₁₃ + e₂₄ to obtain

e₁₂ + e₃₄ + e₁₄ + e₂₃ ≤ 2(e13 + e₁₄ + e₂₃ + e₂₄).

• The above says our solution is at least half the optimum.

Remarks

• A 0.12-approximation algorithm exists.^a

• 0.059-approximation algorithms do not exist unless NP = ZPP.^b

aGoemans & Williamson (1995).

bH˚astad (1997).

Approximability, Unapproximability, and Between

• Some problems have approximation thresholds less than 1.

– knapsack has a threshold of 0 (p. 782).

– node cover (p. 738), bin packing, and maxsat^a have a threshold larger than 0.

• The situation is maximally pessimistic for tsp (p. 757) and independent set,^b which cannot be approximated

– Their approximation threshold is 1.

aWilliamson & Shmoys (2011).

bSee the textbook.

Unapproximability of tsp

Theorem 83 The approximation threshold of tsp is 1 unless P = NP.

• Suppose there is a polynomial-time -approximation algorithm for tsp for some  < 1.

• We shall construct a polynomial-time algorithm to solve the NP-complete hamiltonian cycle.

• Given any graph G = (V, E), construct a tsp with | V | cities with distances

d_ij =

⎧⎨

⎩

1, if [ i, j ] ∈ E,

| V |

1−, otherwise.

aSahni & Gonzales (1976).

The Proof (continued)

• Run the alleged approximation algorithm on this tsp instance.

• Note that if a tour includes edges of length | V |/(1 − ), then the tour costs more than | V |.

• Note also that no tour has a cost less than | V |.

• Suppose a tour of cost | V | is returned.

– Then every edge on the tour exists in the original graph G.

– So this tour is a Hamiltonian cycle on G.

The Proof (concluded)

• Suppose a tour that includes an edge of length

| V |/(1 − ) is returned.

– The total length of this tour exceeds | V |/(1 − ).^a – Because the algorithm is -approximate, the optimum

is at least 1 −  times the returned tour’s length.

– The optimum tour has a cost exceeding | V |.

– Hence G has no Hamiltonian cycles.

aSo this reduction is gap introducing.

metric tsp

• metric tsp is similar to tsp.

• But the distances must satisfy the triangular inequality:

d_ij ≤ d_ik + d_kj for all i, j, k.

• Inductively,

d_ij ≤ d_ik + d_kl + · · · + d_zj.

A 0.5-Approximation Algorithm for metric tsp

• It suffices to present an algorithm with the approximation ratio of

c(M (x))

opt(x) ≤ 2 (see p. 733).

aChoukhmane (1978); Iwainsky, Canuto, Taraszow, & Villa (1986);

Kou, Markowsky, & Berman (1981); Plesn´ık (1981).

A 0.5-Approximation Algorithm for metric tsp (concluded)

1: T := a minimum spanning tree of G;

2: T^ := duplicate the edges of T plus their cost; {Note: T^ is an Eulerian multigraph.}

3: C := an Euler cycle of T^;

4: Remove repeated nodes of C; {“Shortcutting.”}

5: return C;

Analysis

• Let Copt be an optimal tsp tour.

• Note first that

c(T ) ≤ c(Copt). (21) – C_opt is a spanning tree after the removal of one edge.

– But T is a minimum spanning tree.

• Because T^ doubles the edges of T , c(T^) = 2c(T ).

Analysis (concluded)

• Because of the triangular inequality, “shortcutting” does not increase the cost.

– (1, 2, 3, 2, 1, 4, . . .) → (1, 2, 3, 4, . . .), a Hamiltonian cycle.

• Thus

c(C) ≤ c(T^).

• Combine all the inequalities to yield

c(C) ≤ c(T^) = 2c(T ) ≤ 2c(C_opt), as desired.

A 100-Node Example

The cost is 7.72877.

A 100-Node Example (continued)

The minimum spanning tree T .

A 100-Node Example (continued)

“Shortcutting” the repeated nodes on the Euler cycle C.

A 100-Node Example (concluded)

The cost is 10.5718 ≤ 2 × 7.72877 = 15.4576.

A (1/3)-Approximation Algorithm for metric tsp

• It suffices to present an algorithm with the approximation ratio of

c(M (x))

opt(x) ≤ 3 2 (see p. 733).

• This is the best approximation ratio for metric tsp as of 2016!

aChristofides (1976).

A (1/3)-Approximation Algorithm for metric tsp (concluded)

1: T := a minimum spanning tree of G;

2: V ^ := the set of nodes with an odd degree in T ; {| V ^ | must be even by a well-known parity result.}

3: G^ := the induced subgraph of G by V ^; {G^ is a complete graph on V ^.}

4: M := a minimum-cost perfect matching of G^;

5: G^ := T ∪ M; {G^ is an Eulerian multigraph.}

6: C := an Euler cycle of G^;

7: Remove repeated nodes of C; {“Shortcutting.”}

8: return C;

Analysis

• Let C_opt be an optimal tsp tour.

• By Eq. (21) on p. 763,

c(T ) ≤ c(Copt). (22)

• Let C^ be C_opt on V ^ by “shortcutting.”

– C_opt is a Hamiltonian cycle on V .

– Replace any path (v₁, v₂, . . . , v_k) on C_opt with (v₁, v_k), where v₁, v_k ∈ V ^ but v₂, . . . , v_k−1 ∈ V ^.

• So C^ is simply the restriction of C_opt to V ^.

Analysis (continued)

• By the triangular inequality,

c(C^) ≤ c(C_opt).

• C^ is now a Hamiltonian cycle on V ^.

• C^ consists of two perfect matchings on G^.^a – The first, third, . . . edges constitute one.

– The second, fourth, . . . edges constitute the other.

aNote that G^ is a complete graph with an even | V ^|.

Analysis (continued)

• By Eq. (22) on p. 771, the cheaper perfect matching has a cost of

c(C^)

2 ≤ c(C_opt) 2 .

• As a result, the minimum-cost one M must satisfy c(M ) ≤ c(C^)

2 ≤ c(C_opt) 2 .

• Minimum-cost perfect matching can be solved in polynomial time.^a

aEdmonds (1965); Micali & V. Vazirani (1980).

Analysis (concluded)

• By combining the two earlier inequalities, any Euler cycle C has a cost of

c(C) ≤ c(T ) + c(M) by Line 5 of the algorithm

≤ c(Copt) + c(C_opt) 2

= 3

2 c(C_opt), as desired.

A 100-Node Example

The cost is 7.72877.

A 100-Node Example (continued)

A 100-Node Example (continued)

A minimum-cost perfect matching M .

A 100-Node Example (continued)

 ∪ M.

A 100-Node Example (continued)

“Shortcutting” the repeated nodes on the Euler cycle C.

A 100-Node Example (continued)

The cost is 8.74583 ≤ (3/2) × 7.72877 = 11.5932.^a

aIn comparison, the earlier 0.5-approximation algorithm gave a cost of 10.5718 on p. 768.

A 100-Node Example (concluded)

If a different Euler cycle were generated on p. 778, the cost could be different, such as 8.54902 (above), 8.85674, 8.53410, 9.20841, and 8.87152.^a

aContributed by Mr. Yu-Chuan Liu (B00507010, R04922040) on July 15, 2017.

knapsack Has an Approximation Threshold of Zero

Theorem 84 For any , there is a polynomial-time

-approximation algorithm for knapsack.

• We have n weights w1, w₂, . . . , w_n ∈ Z⁺, a weight limit W , and n values v₁, v₂, . . . , v_n ∈ Z⁺.^b

• We must find an I ⊆ { 1, 2, . . . , n } such that



i∈I w_i ≤ W and 

i∈I v_i is the largest possible.

aIbarra & Kim (1975).

bIf the values are fractional, the result is slightly messier, but the main conclusion remains correct. Contributed by Mr. Jr-Ben Tian (B89902011, R93922045) on December 29, 2004.

The Proof (continued)

• Let

V = max{ v₁, v₂, . . . , v_n }.

• Clearly, 

i∈I v_i ≤ nV .

• Let 0 ≤ i ≤ n and 0 ≤ v ≤ nV .

• W (i, v) is the minimum weight attainable by selecting only from the first i items and with a total value of v.

– It is an (n + 1) × (nV + 1) table.

The Proof (continued)

• Set W (0, v) = ∞ for v ∈ { 1, 2, . . . , nV } and W (i, 0) = 0 for i = 0, 1, . . . , n.^a

• Then, for 0 ≤ i < n and 1 ≤ v ≤ nV ,^b W (i + 1, v)

=

⎧⎨

⎩

min{ W (i, v), W (i, v − vi+1) + w_i+1 }, if v ≥ vi+1,

W (i, v), otherwise.

• Finally, pick the largest v such that W (n, v) ≤ W .^c

aContributed by Mr. Ren-Shuo Liu (D98922016) and Mr. Yen-Wei Wu (D98922013) on December 28, 2009.

bThe textbook’s formula has an error here.

v

0 nV

≤ W

The Proof (continued)

With 6 items, values (4, 3, 3, 3, 2, 3), weights (3, 3, 1, 3, 2, 1), and W = 12, the maximum total value 16 is achieved with I = { 1, 2, 3, 4, 6 }; I’s weight is 11.

The Proof (continued)

• The running time O(n²V ) is not polynomial.

• Call the problem instance

x = (w₁, . . . , w_n, W, v₁, . . . , v_n).

• Additional idea: Limit the number of precision bits.

• Define

v_i^ =

 v_i 2^b

 .

• Note that

v_i ≥ 2^bv_i^ > v_i − 2^b.

The Proof (continued)

• Call the approximate instance

x^ = (w₁, . . . , w_n, W, v₁^ , . . . , v_n^ ).

• Solving x^ takes time O(n²V /2^b).

– Use v_i^ = vi/2^b and V ^ = max(v₁^ , v₂^ , . . . , v_n^ ) in the dynamic programming.

– It is now an (n + 1) × (nV + 1)/2^b table.

• The selection I^ is optimal for x^.

• But I^ may not be optimal for x, although it still satisfies the weight budget W .

The Proof (continued)

With the same parameters as p. 786 and b = 1: Values are (2, 1, 1, 1, 1, 1) and the optimal selection I^ = { 1, 2, 3, 5, 6 } for x^ has a smaller maximum value 4 + 3 + 3 + 2 + 3 = 15 for x than I’s 16; its weight is 10 < W = 12.^a

aThe original optimal I = { 1, 2, 3, 4, 6 } on p. 786 has the same value 6 and but higher weight 11 for x^.

The Proof (continued)

• The value of I^ for x is close to that of the optimal I:



i∈I^

v_i ≥ 

i∈I^

2^bv_i^ = 2^b 

i∈I^

v_i^

≥ 2^b 

i∈I

v_i^ = 

i∈I

2^bv_i^

≥ 

i∈I

v_i − 2^b

≥



i∈I

v_i 

− n2^b.

The Proof (continued)

• In summary,



i∈I^

v_i ≥



i∈I

v_i 

− n2^b.

• Without loss of generality, assume wi ≤ W for all i.

– Otherwise, item i is redundant and can be removed early on.

• V is a lower bound on opt.

– Picking one single item with value V is a legitimate choice.

The Proof (concluded)

• The relative error from the optimum is:



i∈I v_i − 

i∈I^ v_i



i∈I v_i ≤



i∈I v_i − 

i∈I^ v_i

V ≤ n2^b

V .

• Suppose we pick b = log₂ ^V_n .

• The algorithm becomes -approximate.^a

• The running time is then O(n²V /2^b) = O(n³/), a polynomial in n and 1/.^b

aSee Eq. (17) on p. 727.

bIt hence depends on the value of 1/. Thanks to a lively class dis- cussion on December 20, 2006. If we fix  and let the problem size increase, then the complexity is cubic. Contributed by Mr. Ren-Shan

Comments

• independent set and node cover are reducible to each other (Corollary 45, p. 375).

• node cover has an approximation threshold at most 0.5 (p. 740).

• But independent set is unapproximable (see the textbook).

• independent set limited to graphs with degree ≤ k is called k-degree independent set.

• k-degree independent set is approximable (see the textbook).

On P vs. NP

If 50 million people believe a foolish thing, it’s still a foolish thing.

— George Bernard Shaw (1856–1950)

Exponential Circuit Complexity for NP-Complete Problems

• We shall prove exponential lower bounds for NP-complete problems using monotone circuits.

– Monotone circuits are circuits without ¬ gates.^a

• Note that this result does not settle the P vs. NP problem.

aRecall p. 313.

The Power of Monotone Circuits

• Monotone circuits can only compute monotone boolean functions.

• They are powerful enough to solve a P-complete problem: monotone circuit value (p. 314).

• There are NP-complete problems that are not monotone;

they cannot be computed by monotone circuits at all.

• There are NP-complete problems that are monotone;

they can be computed by monotone circuits.

– hamiltonian path and clique.

clique

• clique_n,k is the boolean function deciding whether a graph G = (V, E) with n nodes has a clique of size k.

• The input gates are the _n

2

entries of the adjacency matrix of G.

– Gate g_ij is set to true if the associated undirected edge { i, j } exists.

• clique_n,k is a monotone function.

• Thus it can be computed by a monotone circuit.

• This does not rule out that nonmonotone circuits for clique_n,k may use fewer gates.

Crude Circuits

• One possible circuit for clique_n,k does the following.

1. For each S ⊆ V with | S | = k, there is a circuit with O(k²) ∧-gates testing whether S forms a clique.

2. We then take an or of the outcomes of all the _n

k

subsets S₁, S₂, . . . , S(ⁿ_k).

• This is a monotone circuit with O(k² _n

k

) gates, which is exponentially large unless k or n − k is a constant.

• A crude circuit CC(X₁, X₂, . . . , X_m) tests if there is an X_i ⊆ V that forms a clique.

– The above-mentioned circuit is CC(S₁, S₂, . . . , S(ⁿ_k)).

The Proof: Positive Examples

• Analysis will be applied to only the following positive examples and negative examples as input graphs.

• A positive example is a graph that has _k

2

edges connecting k nodes in all possible ways.

• There are _n

k

such graphs.

• They all should elicit a true output from clique_n,k.

The Proof: Negative Examples

• Color the nodes with k − 1 different colors and join by an edge any two nodes that are colored differently.

• There are (k − 1)ⁿ such graphs.

• They all should elicit a false output from clique_n,k. – Each set of k nodes must have 2 identically colored

nodes; hence there is no edge between them.

Positive and Negative Examples with k = 5

$ SRVLWLYH H[DPSOH $ QHJDWLYH H[DPSOH