### max cut Revisited

• The NP-complete max cut seeks to partition the nodes of graph G = (V, E) into (S, V − S) so that there are as many edges as possible between S and V − S (p. 290).

• Local search starts from a feasible solution and

performs “local” improvements until none are possible.

### A 0.5-Approximation Algorithm for max cut

1: S := ∅;

2: while ∃v ∈ V whose switching sides results in a larger cut do

3: Switch the side of v;

4: end while

5: return S;

• A 0.12-approximation algorithm exists.^{a}

• 0.059-approximation algorithms do not exist unless NP = ZPP.

aGoemans and Williamson (1995).

### Analysis

*V*_{3} *V*_{4}

*V*_{2}
*V*_{1}

Optimal cut

Our cut

*e*_{12}

*e*_{13}

*e*_{24}

*e*_{34}
*e*_{14} *e*_{23}

### Analysis (continued)

• Partition V = V^{1} ∪ V^{2} ∪ V^{3} ∪ V^{4}, where our algorithm
returns (V_{1} ∪ V^{2}, V_{3} ∪ V^{4}) and the optimum cut is

(V_{1} ∪ V^{3}, V_{2} ∪ V^{4}).

• Let e^{ij} be the number of edges between V_{i} and V_{j}.

• Because no migration of nodes can improve the

algorithm’s cut, for each node in V_{1}, its edges to V_{1} ∪ V^{2}
are outnumbered by those to V_{3} ∪ V^{4}.

• Considering all nodes in V^{1} together, we have
2e_{11} + e_{12} ≤ e^{13} + e_{14}, which implies

e_{12} ≤ e13 + e_{14}.

### Analysis (concluded)

• Similarly,

e_{12} ≤ e^{23} + e_{24}
e_{34} ≤ e^{23} + e_{13}
e_{34} ≤ e^{14} + e_{24}

• Adding all four inequalities, dividing both sides by 2, and adding the inequality

e_{14} + e_{23} ≤ e^{14} + e_{23} + e_{13} + e_{24}, we obtain

e_{12} + e_{34} + e_{14} + e_{23} ≤ 2(e^{13} + e_{14} + e_{23} + e_{24}).

• The above says our solution is at least half the optimum.

### Approximability, Unapproximability, and Between

• knapsack, node cover, maxsat, and max cut have approximation thresholds less than 1.

– knapsack has a threshold of 0 (see p. 590).

– But node cover and maxsat have a threshold larger than 0.

• The situation is maximally pessimistic for tsp: It cannot be approximated unless P = NP (see p. 588).

– The approximation threshold of tsp is 1.

∗ The threshold is 1/3 if the tsp satisfies the triangular inequality.

– The same holds for independent set.

### Unapproximability of tsp

^{a}

Theorem 74 The approximation threshold of tsp is 1 unless P = NP.

• Suppose there is a polynomial-time ǫ-approximation algorithm for tsp for some ǫ < 1.

• We shall construct a polynomial-time algorithm for the NP-complete hamiltonian cycle.

• Given any graph G = (V, E), construct a tsp with |V | cities with distances

d_{ij} =

1, if { i, j } ∈ E

|V |

1−ǫ, otherwise

aSahni and Gonzales (1976).

### The Proof (concluded)

• Run the alleged approximation algorithm on this tsp.

• Suppose a tour of cost |V | is returned.

– This tour must be a Hamiltonian cycle.

• Suppose a tour with at least one edge of length _{1−ǫ}^{|V |} is
returned.

– The total length of this tour is > _{1−ǫ}^{|V |} .

– Because the algorithm is ǫ-approximate, the optimum is at least 1 − ǫ times the returned tour’s length.

– The optimum tour has a cost exceeding | V |.

– Hence G has no Hamiltonian cycles.

### knapsack Has an Approximation Threshold of Zero

^{a}

Theorem 75 For any ǫ, there is a polynomial-time ǫ-approximation algorithm for knapsack.

• We have n weights w1, w_{2}, . . . , w_{n} ∈ Z^{+}, a weight limit
W , and n values v_{1}, v_{2}, . . . , vn ∈ Z^{+}.^{b}

• We must find an S ⊆ {1, 2, . . . , n} such that P

i∈S w_{i} ≤ W and P

i∈S v_{i} is the largest possible.

• Let

V = max{v^{1}, v_{2}, . . . , vn}.

aIbarra and Kim (1975).

bIf the values are fractional, the result is slightly messier but the main conclusion remains correct. Contributed by Mr. Jr-Ben Tian (R92922045) on December 29, 2004.

### The Proof (continued)

• For 0 ≤ i ≤ n and 0 ≤ v ≤ nV , define W (i, v) to be the minimum weight attainable by selecting some among the i first items, so that their value is exactly v.

• Start with W (0, v) = ∞ for all v.

• Then

W (i + 1, v) = min{W (i, v), W (i, v − vi+1) + w_{i+1}}.

• Finally, pick the largest v such that W (n, v) ≤ W .

• The running time is O(n^{2}V ), not polynomial time.

• Key idea: Limit the number of precision bits.

### The Proof (continued)

• Given the instance x = (w^{1}, . . . , wn, W, v_{1}, . . . , vn), we
define the approximate instance

x^{′} = (w_{1}, . . . , w_{n}, W, v_{1}^{′} , . . . , v_{n}^{′} ),
where

v_{i}^{′} = 2^{b} j vi

2^{b}
k.

• Solving x^{′} takes time O(n^{2}V /2^{b}).

• The solution S^{′} is close to the optimum solution S:

X

i∈S

v_{i} ≥ X

i∈S^{′}

v_{i} ≥ X

i∈S^{′}

v_{i}^{′} ≥ X

i∈S

v_{i}^{′} ≥ X

i∈S

(v^{i} − 2^{b}) ≥ X

i∈S

v_{i} − n2^{b}.

### The Proof (continued)

• Hence

X

i∈S^{′}

v_{i} ≥ X

i∈S

v_{i} − n2^{b}.

• Without loss of generality, w^{i} ≤ W (otherwise item i is
redundant).

• V is a lower bound on opt.

– Picking the item with value V alone is a legitimate choice.

• The relative error from the optimum is ≤ n2^{b}/V as
P

i∈S vi − P

i∈S^{′} vi

P

i∈S v_{i} ≤

P

i∈S vi − P

i∈S^{′} vi

V ≤ n2^{b}

V .

### The Proof (concluded)

• Truncate the last b = ⌊log2 ǫV

n ⌋ bits of the values.

• The algorithm becomes ǫ-approximate (see Eq. (8) on p. 567).

• The running time is then O(n^{2}V /2^{b}) = O(n^{3}/ǫ), a
polynomial in n and 1/ǫ.

### Pseudo-Polynomial-Time Algorithms

• Consider problems with inputs that consist of a

collection of integer parameters (tsp, knapsack, etc.).

• An algorithm for such a problem whose running time is a polynomial of the input length and the value (not length) of the largest integer parameter is a

pseudo-polynomial-time algorithm.^{a}

• On p. 591, we presented a pseudo-polynomial-time
algorithm for knapsack that runs in time O(n^{2}V ).

• How about tsp (d), another NP-complete problem?

aGarey and Johnson (1978).

### No Pseudo-Polynomial-Time Algorithms for tsp (d)

• By definition, a pseudo-polynomial-time algorithm becomes polynomial-time if each integer parameter is limited to having a value polynomial in the input length.

• Corollary 38 (p. 306) showed that hamiltonian path is reducible to tsp (d) with weights 1 and 2.

• As hamiltonian path is NP-complete, tsp (d) cannot have pseudo-polynomial-time algorithms unless P = NP.

• tsp (d) is said to be strongly NP-hard.

• Many weighted versions of NP-complete problems are strongly NP-hard.

### Polynomial-Time Approximation Scheme

• Algorithm M is a polynomial-time approximation scheme (PTAS) for a problem if:

– For each ǫ > 0 and instance x of the problem, M runs in time polynomial (depending on ǫ) in | x |.

∗ Think of ǫ as a constant.

– M is an ǫ-approximation algorithm for every ǫ > 0.

### Fully Polynomial-Time Approximation Scheme

• A polynomial-time approximation scheme is fully polynomial (FPTAS) if the running time depends polynomially on | x | and 1/ǫ.

– Maybe the best result for a “hard” problem.

– For instance, knapsack is fully polynomial with a
running time of O(n^{3}/ǫ) (p. 590).

### Square of G

• Let G = (V, E) be an undirected graph.

• G^{2} has nodes {(v^{1}, v_{2}) : v_{1}, v_{2} ∈ V } and edges

{{ (u, u^{′}), (v, v^{′})} : (u = v ∧ { u^{′}, v^{′} } ∈ E) ∨ { u, v } ∈ E}.

1

2

3

(1,1)

*G*

(1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

*G*^{2}

### Independent Sets of G and G

^{2}

Lemma 76 G(V, E) has an independent set of size k if and
only if G^{2} has an independent set of size k^{2}.

• Suppose G has an independent set I ⊆ V of size k.

• {(u, v) : u, v ∈ I} is an independent set of size k^{2} of G^{2}.

1

2

3

(1,1)

*G*

(1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

*G*^{2}

### The Proof (continued)

• Suppose G^{2} has an independent set I^{2} of size k^{2}.

• U ≡ {u : ∃v ∈ V (u, v) ∈ I^{2}} is an independent set of G.

1

2

3

(1,1)

*G*

(1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

*G*^{2}

• | U | is the number of “rows” that the nodes in I^{2} occupy.

### The Proof (concluded)

^{a}

• If | U | ≥ k, then we are done.

• Now assume | U | < k.

• As the k^{2} nodes in I^{2} cover fewer than k “rows,” there
must be a “row” in possession of > k nodes of I^{2}.

• Those > k nodes will be independent in G as each “row”

is a copy of G.

aThanks to a lively class discussion on December 29, 2004.

### Approximability of independent set

• The approximation threshold of the maximum independent set is either zero or one (it is one!).

Theorem 77 If there is a polynomial-time ǫ-approximation algorithm for independent set for any 0 < ǫ < 1, then there is a polynomial-time approximation scheme.

• Let G be a graph with a maximum independent set of size k.

• Suppose there is an O(n^{i})-time ǫ-approximation
algorithm for independent set.

### The Proof (continued)

• By Lemma 76 (p. 600), the maximum independent set of
G^{2} has size k^{2}.

• Apply the algorithm to G^{2}.

• The running time is O(n^{2i}).

• The resulting independent set has size ≥ (1 − ǫ) k^{2}.

• By the construction in Lemma 76 (p. 600), we can obtain an independent set of size ≥ p

(1 − ǫ) k^{2} for G.

• Hence there is a (1 − √

1 − ǫ)-approximation algorithm for independent set.

### The Proof (concluded)

• In general, we can apply the algorithm to G^{2}^{ℓ} to obtain
an (1 − (1 − ǫ)^{2}^{−ℓ})-approximation algorithm for

independent set.

• The running time is n^{2}^{ℓ}^{i}.^{a}

• Now pick ℓ = ⌈log _{log(1−ǫ}^{log(1−ǫ)}^{′}_{)}⌉.

• The running time becomes n^{i}^{log(1−ǫ′)}^{log(1−ǫ)} .

• It is an ǫ^{′}-approximation algorithm for independent
set.

aIt is not fully polynomial.

### Comments

• independent set and node cover are reducible to each other (Corollary 36, p. 286).

• node cover has an approximation threshold at most 0.5 (p. 573).

• But independent set is unapproximable (see the textbook).

• independent set limited to graphs with degree ≤ k is called k-degree independent set.

• k-degree independent set is approximable (see the textbook).

## On P vs NP

### Density

^{a}

The density of language L ⊆ Σ^{∗} is defined as
dens_{L}(n) = |{x ∈ L : | x | ≤ n}|.

• If L = {0, 1}^{∗}, then dens_{L}(n) = 2^{n+1} − 1.

• So the density function grows at most exponentially.

• For a unary language L ⊆ {0}^{∗},

dens_{L}(n) ≤ n + 1.

– Because L ⊆ {ǫ, 0, 00, . . . ,

n

z }| {

00· · · 0, . . .}.

aBerman and Hartmanis (1977).

### Sparsity

• Sparse languages are languages with polynomially bounded density functions.

• Dense languages are languages with superpolynomial density functions.

### Self-Reducibility for sat

• An algorithm exploits self-reducibility if it reduces the problem to the same problem with a smaller size.

• Let φ be a boolean expression in n variables
x_{1}, x_{2}, . . . , xn.

• t ∈ {0, 1}^{j} is a partial truth assignment for
x_{1}, x_{2}, . . . , x_{j}.

• φ[ t ] denotes the expression after substituting the truth
values of t for x_{1}, x_{2}, . . . , x_{t} in φ.

### An Algorithm for sat with Self-Reduction

We call the algorithm below with empty t.

1: if | t | = n then

2: return φ[ t ];

3: else

4: return φ[ t0 ] ∨ φ[ t1 ];

5: end if

The above algorithm runs in exponential time, by visiting all the partial assignments (or nodes on a depth-n binary tree).

### NP-Completeness and Density

^{a}

Theorem 78 If a unary language U ⊆ {0}^{∗} is
NP-complete, then P = NP.

• Suppose there is a reduction R from sat to U.

• We shall use R to guide us in finding the truth

assignment that satisfies a given boolean expression φ with n variables if it is satisfiable.

• Specifically, we use R to prune the exponential-time exhaustive search on p. 611.

• The trick is to keep the already discovered results φ[ t ] in a table H.

aBerman (1978).

1: if | t | = n then

2: return φ[ t ];

3: else

4: if (R(φ[ t ]), v) is in table H then

5: return v;

6: else

7: if φ[ t0 ] = “satisfiable” or φ[ t1 ] = “satisfiable” then

8: Insert (R(φ[ t ]), 1) into H;

9: return “satisfiable”;

10: else

11: Insert (R(φ[ t ]), 0) into H;

12: return “unsatisfiable”;

13: end if

14: end if

15: end if

### The Proof (continued)

• Since R is a reduction, R(φ[ t ]) = R(φ[ t^{′} ]) implies that
φ[ t ] and φ[ t^{′} ] must be both satisfiable or unsatisfiable.

• R(φ[ t ]) has polynomial length ≤ p(n) because R runs in log space.

• As R maps to unary numbers, there are only polynomially many p(n) values of R(φ[ t ]).

• How many nodes of the complete binary tree (of invocations/truth assignments) need to be visited?

• If that number is a polynomial, the overall algorithm runs in polynomial time and we are done.

### The Proof (continued)

• A search of the table takes time O(p(n)) in the random access memory model.

• The running time is O(Mp(n)), where M is the total number of invocations of the algorithm.

• The invocations of the algorithm form a binary tree of depth at most n.

### The Proof (continued)

• There is a set T = {t^{1}, t_{2}, . . .} of invocations (partial
truth assignments, i.e.) such that:

– |T | ≥ (M − 1)/(2n).

– All invocations in T are recursive (nonleaves).

– None of the elements of T is a prefix of another.

### VWVWHS'HOHWH OHDYHV0−

### QRQOHDYHVUHPDLQLQJ

### QGVWHS6HOHFWDQ\

### ERWWRPXQGHOHWHG LQYRFDWLRQWDQGDGG LWWR7

### UGVWHS'HOHWHDOOWV DWPRVWQDQFHVWRUV SUHIL[HVIURP

### IXUWKHUFRQVLGHUDWLRQ

### The Proof (continued)

• All invocations t ∈ T have different R(φ[ t ]) values.

– None of s, t ∈ T is a prefix of another.

– The invocation of one started after the invocation of the other had terminated.

– If they had the same value, the one that was invoked second would have looked it up, and therefore would not be recursive, a contradiction.

• The existence of T implies that there are at least (M − 1)/(2n) different R(φ[ t ]) values in the table.

### The Proof (concluded)

• We already know that there are at most p(n) such values.

• Hence (M − 1)/(2n) ≤ p(n).

• Thus M ≤ 2np(n) + 1.

• The running time is therefore O(Mp(n)) = O(np^{2}(n)).

• We comment that this theorem holds for any sparse
language, not just unary ones.^{a}

aMahaney (1980).