A 0.5-Approximation Algorithm

node cover

• node cover seeks the smallest C ⊆ V in graph

G = (V, E) such that for each edge in E, at least one of its endpoints is in C.

• A heuristic to obtain a good node cover is to iteratively move a node with the highest degree to the cover.

• This turns out to produce an approximation ratio of^a c(M (x))

opt(x) = Θ(log n).

• So it is not an -approximation algorithm for any constant  < 1 according to Eq. (19).

aChv´atal (1979).

A 0.5-Approximation Algorithm

1: C := ∅;

2: while E = ∅ do

3: Delete an arbitrary edge { u, v } from E;

4: Add u and v to C; {Add 2 nodes to C each time.}

5: Delete edges incident with u or v from E;

6: end while

7: return C;

aJohnson (1974).

Analysis

• It is easy to see that C is a node cover.

• C contains |C|/2 edges.^a

• No two edges of C share a node.^b

• Any node cover must contain at least one node from each of these edges.

– If there is an edge in C both of whose ends are

outside the cover, then that cover will not be valid.

aThe edges deleted in Line 3.

bIn fact, C as a set of edges is a maximal matching.

Analysis (concluded)

• This means that opt(G) ≥ |C|/2.

• The approximation ratio is hence

|C|

opt(G) ≤ 2.

• So we have a 0.5-approximation algorithm.

• And the approximation threshold is therefore ≤ 0.5.

The 0.5 Bound Is Tight for the Algorithm

Optimal cover

aContributed by Mr. Jenq-Chung Li (R92922087) on December 20, 2003. Recall that K¨onig’s theorem says the size of a maximum matching

Remarks

• The approximation threshold is at least^a 1 − 

10√

5 − 21₋₁

≈ 0.2651.

• The approximation threshold is 0.5 if one assumes the unique games conjecture.^b

• This ratio 0.5 is also the lower bound for any “greedy”

algorithms.^c

aDinur and Safra (2002).

bKhot and Regev (2008).

cDavis and Impagliazzo (2004).

Maximum Satisfiability

• Given a set of clauses, maxsat seeks the truth assignment that satisfies the most.

• max2sat is already NP-complete (p. 347), so maxsat is NP-complete.

• Consider the more general k-maxgsat for constant k.

– Let Φ = {φ₁, φ₂, . . . , φ_m} be a set of boolean expressions in n variables.

– Each φ_i is a general expression involving up to k variables.

– k-maxgsat seeks the truth assignment that satisfies the most expressions.

A Probabilistic Interpretation of an Algorithm

• Let φ_i involve k_i ≤ k variables and be satisfied by s_i of the 2^ki truth assignments.

• A random truth assignment ∈ { 0, 1 }ⁿ satisfies φ_i with probability p(φ_i) = s_i/2^ki.

– p(φ_i) is easy to calculate as k is a constant.

• Hence a random truth assignment satisfies an average of p(Φ) =

m i=1

p(φ_i) expressions φi.

The Search Procedure

• Clearly

p(Φ) = 1

2 { p(Φ[ x1 = true ]) + p(Φ[ x1 = false ]) }.

• Select the t₁ ∈ {true, false} such that p(Φ[ x₁ = t₁ ]) is the larger one.

• Note that p(Φ[ x1 = t₁ ]) ≥ p(Φ).

• Repeat the procedure with expression Φ[ x1 = t1 ] until all variables xi have been given truth values ti and all φi

are either true or false.

The Search Procedure (continued)

• By our hill-climbing procedure, p(Φ)

≤ p(Φ[ x1 = t1 ])

≤ p(Φ[ x1 = t1, x2 = t2 ])

≤ · · ·

≤ p(Φ[ x₁ = t₁, x₂ = t₂, . . . , x_n = t_n ]).

• So at least p(Φ) expressions are satisfied by truth assignment (t₁, t₂, . . . , t_n).

The Search Procedure (concluded)

• Note that the algorithm is deterministic!

• It is called the method of conditional expectations.^a

aErd˝os and Selfridge (1973); Spencer (1987).

Approximation Analysis

• The optimum is at most the number of satisfiable φ_i—i.e., those with p(φ_i) > 0.

• Hence the ratio of algorithm’s output vs. the optimum is^a

≥  p(Φ)

p(φi)>0 1 =



i p(φ_i)



p(φi)>0 1 ≥ min

p(φi)>0p(φi).

• So this is a polynomial-time -approximation algorithm with  = 1 − min_p(φi)>0 p(φi).

• Because p(φi) ≥ 2^−k for a satisfiable φi, the heuristic is a polynomial-time -approximation algorithm with

 = 1 − 2^−k.

aRecall that 

i a_i/

i b_i ≥ min_i(a_i/b_i).

Back to maxsat

• In maxsat, the φi’s are clauses (like x ∨ y ∨ ¬z).

• Hence p(φ_i) ≥ 1/2, which happens when φ_i contains a single literal.

• The heuristic becomes a polynomial-time

-approximation algorithm with  = 1/2.^a

• Suppose we set each boolean variable to true with probability (√

5 − 1)/2, the golden ratio.

• Then follow through the method of conditional expectations to derandomize it.

aJohnson (1974).

Back to maxsat (concluded)

• We will obtain a [ (3 − √

5 ) ]/2-approximation algorithm.^a

– Note [ (3 − √

5 ) ]/2 ≈ 0.382.

• If the clauses have k distinct literals, p(φ_i) = 1 − 2^−k.

• The heuristic becomes a polynomial-time

-approximation algorithm with  = 2^−k.

– This is the best possible for k ≥ 3 unless P = NP.

aLieberherr and Specker (1981).

max cut Revisited

• max cut seeks to partition the nodes of graph

G = (V, E) into (S, V − S) so that there are as many edges as possible between S and V − S.

• It is NP-complete.^a

• Local search starts from a feasible solution and

performs “local” improvements until none are possible.

• Next we present a local-search algorithm for max cut.

aRecall p. 378.

A 0.5-Approximation Algorithm for max cut

1: S := ∅;

2: while ∃v ∈ V whose switching sides results in a larger cut do

3: Switch the side of v;

4: end while

5: return S;

• A 0.12-approximation algorithm exists.^a

• 0.059-approximation algorithms do not exist unless NP = ZPP.

aGoemans and Williamson (1995).

Analysis

V₃ V₄

V₂ V₁

Optimal cut

Our cut

e₁₂

e₁₃

e₂₄

e₃₄ e₁₄ e₂₃

Analysis (continued)

• Partition V = V₁ ∪ V₂ ∪ V₃ ∪ V₄, where

– Our algorithm returns (V1 ∪ V2, V3 ∪ V4).

– The optimum cut is (V₁ ∪ V3, V₂ ∪ V4).

• Let eij be the number of edges between Vi and Vj.

• Our algorithm returns a cut of size

e₁₃ + e₁₄ + e₂₃ + e₂₄.

• The optimum cut size is

e₁₂ + e₃₄ + e₁₄ + e₂₃.

Analysis (continued)

• For each node v ∈ V₁, its edges to V₁ ∪ V₂ are outnumbered by those to V₃ ∪ V₄.

– Otherwise, v would have been moved to V₃ ∪ V4 to improve the cut.

• Considering all nodes in V1 together, we have 2e11 + e12 ≤ e13 + e14.

– 2e₁₁, because each edge in V₁ is counted twice.

• The above inequality implies

e12 ≤ e13 + e14.

Analysis (concluded)

• Similarly,

e12 ≤ e23 + e24

e₃₄ ≤ e₂₃ + e₁₃ e₃₄ ≤ e₁₄ + e₂₄

• Add all four inequalities, divide both sides by 2, and add the inequality e₁₄ + e₂₃ ≤ e₁₄ + e₂₃ + e₁₃ + e₂₄ to obtain

e₁₂ + e₃₄ + e₁₄ + e₂₃ ≤ 2(e13 + e₁₄ + e₂₃ + e₂₄).

• The above says our solution is at least half the optimum.

Approximability, Unapproximability, and Between

• knapsack, node cover, maxsat, and max cut have approximation thresholds less than 1.

– knapsack has a threshold of 0 (p. 745).

– But node cover (p. 725) and maxsat have a threshold larger than 0.

• The situation is maximally pessimistic for tsp, which cannot be approximated (p. 743).

– The approximation threshold of tsp is 1.

∗ The threshold is 1/3 if tsp satisfies the triangular inequality.

– The same holds for independent set (see the textbook).

Unapproximability of tsp

Theorem 85 The approximation threshold of tsp is 1 unless P = NP.

• Suppose there is a polynomial-time -approximation algorithm for tsp for some  < 1.

• We shall construct a polynomial-time algorithm to solve the NP-complete hamiltonian cycle.

• Given any graph G = (V, E), construct a tsp with | V | cities with distances

d_ij =

⎧⎨

⎩

1, if { i, j } ∈ E

| V |

1−, otherwise

aSahni and Gonzales (1976).

The Proof (concluded)

• Run the alleged approximation algorithm on this tsp.

• Suppose a tour of cost | V | is returned.

– This tour must be a Hamiltonian cycle.

• Suppose a tour that includes an edge of length ^{| V |}_1− is returned.

– The total length of this tour is > ^{| V |}_1−.

– Because the algorithm is -approximate, the optimum is at least 1 −  times the returned tour’s length.

– The optimum tour has a cost exceeding | V |.

– Hence G has no Hamiltonian cycles.

knapsack Has an Approximation Threshold of Zero

Theorem 86 For any , there is a polynomial-time

-approximation algorithm for knapsack.

• We have n weights w1, w2, . . . , wn ∈ Z⁺, a weight limit W , and n values v1, v2, . . . , vn ∈ Z⁺.^b

• We must find an I ⊆ {1, 2, . . . , n} such that



i∈I wi ≤ W and 

i∈I vi is the largest possible.

aIbarra and Kim (1975).

bIf the values are fractional, the result is slightly messier, but the main conclusion remains correct. Contributed by Mr. Jr-Ben Tian (B89902011, R93922045) on December 29, 2004.

The Proof (continued)

• Let

V = max{v₁, v₂, . . . , v_n}.

• Clearly, 

i∈I vi ≤ nV .

• Let 0 ≤ i ≤ n and 0 ≤ v ≤ nV .

• W (i, v) is the minimum weight attainable by selecting only from the first i items and with a total value of v.

– It is an (n + 1) × (nV + 1) table.

The Proof (continued)

• Set W (0, v) = ∞ for v ∈ { 1, 2, . . . , nV } and W (i, 0) = 0 for i = 0, 1, . . . , n.^a

• Then, for 0 ≤ i < n,

W (i + 1, v) = min{W (i, v), W (i, v − vi+1) + wi+1}.

• Finally, pick the largest v such that W (n, v) ≤ W .^b

• The running time is O(n²V ), not polynomial time.

• Key idea: Limit the number of precision bits.

aContributed by Mr. Ren-Shuo Liu (D98922016) and Mr. Yen-Wei Wu (D98922013) on December 28, 2009.

bLawler (1979).

v

<W

nV

The Proof (continued)

• Define

v_i^ = 2^b

vi

2^b

.

– This is equivalent to zeroing each vi’s last b bits.

• Call the original instance

x = (w₁, . . . , w_n, W, v₁, . . . , v_n).

• Call the approximate instance

x^ = (w₁, . . . , w_n, W, v₁^ , . . . , v_n^ ).

The Proof (continued)

• Solving x^ takes time O(n²V /2^b).

– The algorithm only performs subtractions on the vi-related values.

– So the b last bits can be removed from the calculations.

– That is, use v_i^ = _vi

2^b

 and V ^ = max(v₁^, v₂^, . . . , v_n^) in dynamic programming.

– It is now an (n + 1) × (nV + 1)/2^b table.

– Then multiply the returned value by 2^b.

• The selection I^ is optimal for x^.

The Proof (continued)

• The selection I^ is close to the optimal selection I, for x:



i∈I^

v_i ≥ 

i∈I^

v_i^ ≥ 

i∈I

v_i^ ≥ 

i∈I

(v_i − 2^b) ≥



i∈I

v_i



− n2^b.

• Hence



i∈I^

vi ≥



i∈I

vi



− n2^b.

• Without loss of generality, assume w_i ≤ W for all i.

– Otherwise, item i is redundant.

• V is a lower bound on opt.

– Picking an item with value V is a legitimate choice.

The Proof (concluded)

• The relative error from the optimum is:



i∈I vi − 

i∈I^ vi



i∈I v_i ≤



i∈I vi − 

i∈I^ vi

V ≤ n2^b

V .

• Suppose we pick b = log₂ ^V_n .

• The algorithm becomes -approximate.^a

• The running time is then O(n²V /2^b) = O(n³/), a polynomial in n and 1/.^b

aSee Eq. (17) on p. 715.

bIt hence depends on the value of 1/. Thanks to a lively class dis- cussion on December 20, 2006. If we fix  and let the problem size increase, then the complexity is cubic. Contributed by Mr. Ren-Shan Luoh (D97922014) on December 23, 2008.

Comments

• independent set and node cover are reducible to each other (Corollary 45, p. 371).

• node cover has an approximation threshold at most 0.5 (p. 727).

• But independent set is unapproximable (see the textbook).

• independent set limited to graphs with degree ≤ k is called k-degree independent set.

• k-degree independent set is approximable (see the textbook).

On P vs. NP

If 50 million people believe a foolish thing, it’s still a foolish thing.

— George Bernard Shaw (1856–1950)

Density

The density of language L ⊆ Σ^∗ is defined as dens_L(n) = |{x ∈ L : | x | ≤ n}|.

• If L = { 0, 1 }^∗, then dens_L(n) = 2ⁿ⁺¹ − 1.

• So the density function grows at most exponentially.

• For a unary language L ⊆ { 0 }^∗,

dens_L(n) ≤ n + 1.

– Because L ⊆ { , 0, 00, . . . ,

  n

00 · · · 0, . . . }.

aBerman and Hartmanis (1977).

Sparsity

• Sparse languages are languages with polynomially bounded density functions.

• Dense languages are languages with superpolynomial density functions.

Self-Reducibility for sat

• An algorithm exhibits self-reducibility if it finds a certificate by exploiting algorithms for the decision version of the same problem.

• Let φ be a boolean expression in n variables x1, x2, . . . , xn.

• t ∈ { 0, 1 }^j is a partial truth assignment for x₁, x₂, . . . , x_j.

• φ[ t ] denotes the expression after substituting the truth values of t for x₁, x₂, . . . , x_{| t |} in φ.

An Algorithm for sat with Self-Reduction

We call the algorithm below with empty t.

1: if | t | = n then

2: return φ[ t ];

3: else

4: return φ[ t0 ] ∨ φ[ t1 ];

5: end if

The above algorithm runs in exponential time, by visiting all the partial assignments (or nodes on a depth-n binary tree).^a

aThe same idea was used in the proof of Proposition 79 on p. 614.

NP-Completeness and Density

Theorem 87 If a unary language U ⊆ { 0 }^∗ is NP-complete, then P = NP.

• Suppose there is a reduction R from sat to U.

• We use R to find a truth assignment that satisfies

boolean expression φ with n variables if it is satisfiable.

• Specifically, we use R to prune the exponential-time exhaustive search on p. 759.

• The trick is to keep the already discovered results φ[ t ] in a table H.

aBerman (1978).

1: if | t | = n then

2: return φ[ t ];

3: else

4: if (R(φ[ t ]), v) is in table H then

5: return v;

6: else

7: if φ[ t0 ] = “satisfiable” or φ[ t1 ] = “satisfiable” then

8: Insert (R(φ[ t ]), “satisfiable”) into H;

9: return “satisfiable”;

10: else

11: Insert (R(φ[ t ]), “unsatisfiable”) into H;

12: return “unsatisfiable”;

13: end if

14: end if

15: end if

The Proof (continued)

• Since R is a reduction, R(φ[ t ]) = R(φ[ t^ ]) implies that φ[ t ] and φ[ t^ ] must be both satisfiable or unsatisfiable.

• R(φ[ t ]) has polynomial length ≤ p(n) because R runs in log space.

• As R maps to unary numbers, there are only polynomially many p(n) values of R(φ[ t ]).

• How many nodes of the complete binary tree (of invocations/truth assignments) need to be visited?

The Proof (continued)

• A search of the table takes time O(p(n)) in the random-access memory model.

• The running time is O(Mp(n)), where M is the total number of invocations of the algorithm.

• If that number is a polynomial, the overall algorithm runs in polynomial time and we are done.

• The invocations of the algorithm form a binary tree of depth at most n.

The Proof (continued)

• There is a set T = { t1, t₂, . . . } of invocations^a such that:

1. | T | ≥ (M − 1)/(2n).

2. All invocations in T are recursive (nonleaves).

3. None of the elements of T is a prefix of another.

• To build one such T , carry out the 1st step and then loop over the 2nd and 3rd steps on the next page.

aPartial truth assignments, i.e.

VW VWHS 'HOHWH OHDYHV  − 

QRQOHDYHV UHPDLQLQJ

QG VWHS 6HOHFW DQ\

ERWWRP XQGHOHWHG LQYRFDWLRQ 9 DQG DGG LW WR %

UG VWHS 'HOHWH DOO 9V DW PRVW 3 DQFHVWRUV SUHIL[HV IURP

IXUWKHU FRQVLGHUDWLRQ

An Example

r

a c

d e f

g h i j

l k

1

2 3

4 5

T = { h, j }; none of h and j is a prefix of the other.

The Proof (continued)

• All invocations t ∈ T have different R(φ[ t ]) values.

– The invocation of one started after the invocation of the other had terminated.

– If they had the same value, the one that was invoked later would have looked it up, and therefore would not be recursive, a contradiction.

• The existence of T implies that there are at least (M − 1)/(2n) different R(φ[ t ]) values in the table.

The Proof (concluded)

• We already know that there are at most p(n) such values.

• Hence (M − 1)/(2n) ≤ p(n).

• Thus M ≤ 2np(n) + 1.

• The running time is therefore O(Mp(n)) = O(np²(n)).

Other Results for Sparse Languages

Theorem 88 (Mahaney (1980)) If a sparse language is NP-complete, then P = NP.

Theorem 89 (Fortung (1979)) If a unary language U ⊆ { 0 }^∗ is coNP-complete, then P = NP.

• Suppose there is a reduction R from sat complement to U .

• The rest of the proof is basically identical except that, now, we want to make sure a formula is unsatisfiable.