The Proof (continued)

knapsack

• There is a set of n items.

• Item i has value vi ∈ Z⁺ and weight w_i ∈ Z⁺.

• We are given K ∈ Z⁺ and W ∈ Z⁺.

• knapsack asks if there exists a subset I ⊆ { 1, 2, . . . , n } such that 

i∈I w_i ≤ W and 

i∈I v_i ≥ K.

– We want to achieve the maximum satisfaction within the budget.

knapsack Is NP-Complete

• knapsack ∈ NP: Guess an I and check the constraints.

• We shall reduce x3c to knapsack, in which vi = w_i for all i and K = W .

• The simplified knapsack now asks if a subset of v1, v2, . . . , v_n adds up to exactly K.^b

– Picture yourself as a radio DJ.

aKarp (1972). It can be solved in time O(2^n/2) with space O(2^n/4) (Schroeppel & Shamir, 1981; Vyskoˇc, 1987).

The Proof (continued)

• The primary differences between the two problems are:^a – Sets vs. numbers.

– Union vs. addition.

• We are given a family F = { S1, S2, . . . , S_n } of size-3 subsets of U = { 1, 2, . . . , 3m }.

• x3c asks if there are m sets in F that cover the set U.

– These m subsets are disjoint by necessity.

aThanks to a lively class discussion on November 16, 2010.

The Proof (continued)

• Think of a set as a bit vector^a in { 0, 1 }^3m. – Assume m = 3.

– 110010000 means the set { 1, 2, 5 }.

– 001100010 means the set { 3, 4, 8 }.

• Our goal is

3m

   1 1 · · · 1 .

aCharacteristic vector, so to speak.

The Proof (continued)

• A bit vector can also be seen as a binary number.

• Set union resembles addition:

001100010 + 110010000 111110010

which denotes the set { 1, 2, 3, 4, 5, 8 }, as desired.

The Proof (continued)

• Trouble occurs when there is carry:

010000000 + 010000000 100000000

which denotes the wrong set { 1 }, not the correct { 2 }.

The Proof (continued)

• Or consider

001100010 + 001110000 011010010

which denotes the set { 2, 3, 5, 8 }, not the correct { 3, 4, 5, 8 }.^a

aCorrected by Mr. Chihwei Lin (D97922003) on January 21, 2010.

The Proof (continued)

• Carry may also lead to a situation where we obtain our solution

3m

  

1 1 · · · 1 with more than m sets in F .

• For example, with m = 3,

000100010 001110000 101100000 + 000001101 111111111

The Proof (continued)

• And it uses 4 sets instead of the required m = 3.^a

• To fix this problem, we enlarge the base just enough so that there are no carries.^b

• Because there are n vectors in total, we change the base from 2 to n + 1.

• Every positive integer N has a unique expression in base b: There are b-adic digits 0 ≤ d_i < b such that

N =

k i=0

dibⁱ, dk = 0.

aThanks to a lively class discussion on November 20, 2002.

The Proof (continued)

• Set vi to be the integer corresponding to the bit vector encoding Si in base n + 1:

v_i =^Δ 

j∈S_i

1 × (n + 1)^3m−j (4)

• Set

K =^Δ

3m−1 j=0

1 × (n + 1)^j =

3m

  

1 1 · · · 1 (base n + 1).

• Now in base n + 1, if there is a set I such that

   ^3m

The Proof (continued)

• For example, the case on p. 437 becomes 000100010

001110000 101100000 + 000001101 102311111 in base n + 1 = 6.

• As desired, it no longer meets the goal.

The Proof (continued)

• Suppose F admits an exact cover, say { S1, S2, . . . , S_m }.

• Then picking I = { 1, 2, . . . , m } clearly results in

v1 + v2 + · · · + vm =

3m

   1 1· · · 1 .

• It is important to note that the meaning of addition (+) is independent of the base.^a

– It is just regular addition.

– But an Si may give rise to different integers vi in Eq.

(4) on p. 439 under different bases.

The Proof (concluded)

• On the other hand, suppose there exists an I such that



i∈I

v_i =

3m

   1 1 · · · 1 in base n + 1.

• The no-carry property implies that | I | = m and { Si : i ∈ I }

is an exact cover.

subset sum

Is NP-Complete

• The proof actually proves:

Corollary 52 subset sum is NP-complete.

• The proof can be slightly revised to reduce exact cover to subset sum.

• The proof would not work if you used m + 1 as the base.^b

aRecall p. 431.

bContributed by Mr. Yuchen Wang (R08922157) on November 19,

An Example

• Let m = 3, U = { 1, 2, 3, 4, 5, 6, 7, 8, 9 }, and S1 = { 1, 3, 4 },

S2 = { 2, 3, 4 }, S3 = { 2, 5, 6 }, S4 = { 6, 7, 8 }, S5 = { 7, 8, 9 }.

• Note that n = 5, as there are 5 S_i’s.

• So the base is n + 1 = 6.

An Example (continued)

• Our reduction produces

K =

3×3−1

j=0

6^j =

3×3

  

1 1· · · 16 = 2015539₁₀, v1 = 1011000006 = 1734048₁₀,

v2 = 0111000006 = 334368₁₀, v3 = 0100110006 = 281448₁₀, v4 = 0000011106 = 258₁₀,

000000111

An Example (concluded)

• Note v1 + v3 + v5 = K because

101100000 010011000 + 000000111 111111111

• Indeed,

S1 ∪ S3 ∪ S5 = { 1, 2, 3, 4, 5, 6, 7, 8, 9 }, an exact cover by 3-sets.

bin packing

• We are given N positive integers a1, a2, . . . , aN, an

integer C (the capacity), and an integer B (the number of bins).

• bin packing asks if these numbers can be partitioned into B subsets, each of which has total sum at most C.

• Think of packing bags at the check-out counter.

Theorem 53 bin packing is NP-complete.

bin packing (concluded)

• But suppose a1, a2, . . . , a_N are randomly distributed between 0 and 1.

• Let B be the smallest number of unit-capacity bins capable of holding them.

• Then B can deviate from its average by more than t with probability at most 2e^−2t2/N.^a

aRhee & Talagrand (1987); Dubhashi & Panconesi (2012).

integer programming (ip)

• ip asks whether a system of linear inequalities with integer coefficients has an integer solution.

• In contrast, linear programming (lp) asks whether a system of linear inequalities with integer coefficients has a rational solution.

– lp is solvable in polynomial time.^a

aKhachiyan (1979).

ip Is NP-Complete

• set covering can be expressed by the inequalities Ax ≥ 1, _n

i=1 xi ≤ B, 0 ≤ xi ≤ 1, where – x_i = 1 if and only if S_i is in the cover.

– A is the matrix whose columns are the bit vectors of the sets S1, S2, . . ..

– 1 is the vector of 1s.

– The operations in Ax are standard matrix operations.

– The ith row of Ax is at least 1 means item i is covered.

aKarp (1972); Borosh & Treybig (1976); Papadimitriou (1981).

ip Is NP-Complete (concluded)

• This shows ip is NP-hard.

• Many NP-complete problems can be expressed as an ip problem.

• ip with a fixed number of variables is in P.^a

aLenstra (1983).

Christos Papadimitriou (1949–)

Easier or Harder?

• Adding restrictions on the allowable problem instances will not make a problem harder.

– We are now solving a subset of problem instances or special cases.

– The independent set proof (p. 372) and the knapsack proof (p. 431): equally hard.

– circuit value to monotone circuit value (p. 321): equally hard.

– sat to 2sat (p. 353): easier.

Easier or Harder? (concluded)

• Adding restrictions on the allowable solutions (the solution space) may make a problem harder, equally hard, or easier.

• It is problem dependent.

– min cut to bisection width (p. 405): harder.

– lp to ip (p. 449): harder.

– sat to naesat (p. 365) and max cut to max bisection (p. 403): equally hard.

– 3-coloring to 2-coloring (p. 415): easier.

coNP and Function Problems

I frankly confess I do not know what he means.

— St. Augustin (354–430), City of God (426)

coNP

• By definition, coNP is the class of problems whose complement is in NP.

– L ∈ coNP if and only if ¯L ∈ NP.

• NP problems have succinct certificates.^a

• coNP is therefore the class of problems that have succinct disqualifications:^b

– A “no” instance possesses a short proof of its being a

“no” instance.

– Only “no” instances have such proofs.

coNP (continued)

• Suppose L is a coNP problem.

• There exists a nondeterministic polynomial-time algorithm M such that:

– If x ∈ L, then M (x) = “yes” for all computation paths.

– If x ∈ L, then M (x) = “no” for some computation path.

• If we swap “yes” and “no” in M, the new algorithm decides ¯L ∈ NP in the classic sense (p. 110).

\HV [ ∉ /

\HV QR

\HV QR

\HV [ ∈ /

\HV

\HV

\HV

\HV

coNP (continued)

• So there are 3 major approaches to proving L ∈ coNP.

1. Prove ¯L ∈ NP.

– Especially when you already knew ¯L ∈ NP.

2. Prove that only “no” instances possess short proofs (for their not being in L).^a

3. Write an algorithm for it directly.

aRecall Proposition 41 (p. 335).

coNP (concluded)

• Clearly P ⊆ coNP.

• It is not known if

P = NP ∩ coNP.

– Contrast this with

R = RE ∩ coRE (see p. 159).

Some coNP Problems

• sat complement ∈ coNP.

– sat complement is the complement of sat.^a

– Or, the disqualification is a truth assignment that satisfies it.

• hamiltonian path complement ∈ coNP.

– hamiltonian path complement is the complement of hamiltonian path.

– Or, the disqualification is a Hamiltonian path.

aRecall p. 200.

Some coNP Problems (concluded)

• validity ∈ coNP.

– If φ is not valid, it can be disqualified very succinctly:

a truth assignment that does not satisfy it.

• tsp complement (d) ∈ coNP.

– tsp complement (d) asks if the optimal tour has a total distance of > B, where B is an input.^a

– The disqualification is a tour with a distance ≤ B.

a R95922093) on September 27, 2006.

A Nondeterministic Algorithm for sat complement (See also p. 121)

φ is a boolean formula with n variables.

1: for i = 1, 2, . . . , n do

2: Guess x_i ∈ { 0, 1 }; {Nondeterministic choice.}

3: end for

4: {Verification:}

5: if φ(x1, x2, . . . , xn) = 1 then

6: “no”;

7: else

8: “yes”;

9: end if

Analysis

• The algorithm decides language { φ : φ is unsatisfiable }.

– The computation tree is a complete binary tree of depth n.

– Every computation path corresponds to a particular truth assignment out of 2ⁿ.

– φ is unsatisfiable if and only if every truth assignment falsifies φ.

– But every truth assignment falsifies φ if and only if

An Alternative Characterization of coNP

Proposition 54 Let L ⊆ Σ^∗ be a language. Then L ∈ coNP if and only if there is a polynomially decidable and

polynomially balanced relation R such that L = { x : ∀y (x, y) ∈ R }.

(As on p. 334, we assume | y | ≤ | x |^k for some k.)

• ¯L = { x : ∃y (x, y) ∈ ¬R }.^a

• Because ¬R remains polynomially balanced, ¯L ∈ NP by Proposition 41 (p. 335).

• Hence L ∈ coNP by definition.

coNP-Completeness

Proposition 55 L is NP-complete if and only if its complement ¯L = Σ^∗ − L is coNP-complete.

Proof (⇒; the ⇐ part is symmetric)

• Let L^ be any coNP language.

• Hence L^ ∈ NP.

• Let R be the reduction from L^ to L.

• So x ∈ L^ if and only if R(x) ∈ L.

• By the law of transposition, x ∈ L^ if and only if

coNP Completeness (concluded)

• So x ∈ L^ if and only if R(x) ∈ ¯L.

• The same R is a reduction from L^ to ¯L.

• This shows ¯L is coNP-hard.

• But ¯L ∈ coNP.

• This shows ¯L is coNP-complete.

Some coNP-Complete Problems

• sat complement is coNP-complete.

• hamiltonian path complement is coNP-complete.

• tsp complement (d) is coNP-complete.

• validity is coNP-complete.

– φ is valid if and only if ¬φ is not satisfiable.

– φ ∈ validity if and only if ¬φ ∈ sat complement.

– The reduction from sat complement to validity

Possible Relations between P, NP, coNP

1. P = NP = coNP.

2. NP = coNP but P = NP.

3. NP = coNP and P = NP.

• This is the current “consensus.”^a

aCarl Gauss (1777–1855), “I could easily lay down a multitude of such propositions, which one could neither prove nor dispose of.”

The Primality Problem

• An integer p is prime if p > 1 and all positive numbers other than 1 and p itself cannot divide it.

• primes asks if an integer N is a prime number.

• Dividing N by 2, 3, . . . ,√

N is not efficient.

– The length of N is only log N , but √

N = 2^{0.5 log N}. – It is an exponential-time algorithm.

• A polynomial-time algorithm for primes was not found until 2002 by Agrawal, Kayal, and Saxena!

1: _{if n = a}^b for some a, b > 1 then

2: return “composite”;

3: _{end if}

4: for r = 2, 3, . . . , n − 1 do

5: if gcd(n, r) > 1 then

6: return “composite”;

7: _{end if}

8: if r is a prime then

9: Let q be the largest prime factor of r − 1;

10: _{if q ≥ 4}^√r log n and n^(r−1)/q = 1 mod r then

11: break; {Exit the for-loop.}

12: _{end if} 13: _{end if}

14: end for{r − 1 has a prime factor q ≥ 4√

r log n.}

15: for a = 1, 2, . . . , 2√

r log n do

16: _{if (x − a)}ⁿ _{= (x}ⁿ − a) mod (x^r − 1) in Zn[x ] then

17: return “composite”;

18: _{end if} 19: _{end for}

The Primality Problem (concluded)

• Later, we will focus on efficient “randomized” algorithms for primes (used in Mathematica, e.g.).

• NP ∩ coNP is the class of problems that have succinct certificates and succinct disqualifications.

– Each “yes” instance has a succinct certificate.

– Each “no” instance has a succinct disqualification.

– No instances have both.

• We will see that primes ∈ NP ∩ coNP.

Basic Modular Arithmetics

• Let m, n ∈ Z⁺.

• m | n means m divides n; m is n’s divisor.

• We call the numbers 0, 1, . . . , n − 1 the residue modulo n.

• The greatest common divisor of m and n is denoted gcd(m, n).

aCarl Friedrich Gauss.

Basic Modular Arithmetics (concluded)

• We use

a ≡ b mod n if n | (a − b).

– So 25 ≡ 38 mod 13.

• We use

a = b mod n

if b is the remainder of a divided by n.

Primitive Roots in Finite Fields

Theorem 56 (Lucas & Lehmer, 1927) ^a A number p > 1 is a prime if and only if there is a number 1 < r < p such that

1. r^p−1 = 1 mod p, and

2. r^(p−1)/q = 1 mod p for all prime divisors q of p − 1.

• This r is called a primitive root or generator of p.

• We will prove one direction of the theorem later.^b

aFran¸cois Edouard Anatole Lucas (1842–1891); Derrick Henry Lehmer (1905–1991).

bSee pp. 487ff.

Derrick Lehmer

(1905–1991)

Pratt’s Theorem

Theorem 57 (Pratt, 1975) primes ∈ NP ∩ coNP.

• primes ∈ coNP because a succinct disqualification is a proper divisor.

– A proper divisor of a number means it is not a prime.

• Now suppose p is a prime.

• p’s certificate includes the r in Theorem 56 (p. 476).

– There may be multiple choices for r.

The Proof (continued)

• Use recursive doubling to check if r^p−1 = 1 mod p in time polynomial in the length of the input, log₂ p.

– r, r², r⁴, . . . mod p, a total of ∼ log₂ p steps.

• We also need all prime divisors of p − 1: q1, q2, . . . , q_k. – Whether r, q1, . . . , qk are easy to find is irrelevant.

• Checking r^(p−1)/qi = 1 mod p is also easy.

• Checking q1, q2, . . . , q_k are all the divisors of p − 1 is easy.

The Proof (concluded)

• We still need certificates for the primality of the q_i’s.

• The complete certificate is recursive and tree-like:

C(p) = (r; q1, C(q1), q2, C(q2), . . . , qk, C(qk)). (5)

• We next prove that C(p) is succinct.

• As a result, C(p) can be checked in polynomial time.

A Certificate for 23

• Note that 5 is a primitive root modulo 23 and 23 − 1 = 22 = 2 × 11.^b

• So

C(23) = (5; 2, C(2), 11, C(11)).

• Note that 2 is a primitive root modulo 11 and 11 − 1 = 10 = 2 × 5.

• So

C(11) = (2; 2, C(2), 5, C(5)).

A Certificate for 23 (concluded)

• Note that 2 is a primitive root modulo 5 and 5 − 1 = 4 = 2².

• So

C(5) = (2; 2, C(2)).

• In summary,

C(23) = (5; 2, C(2), 11, (2; 2, C(2), 5, (2; 2, C(2)))).

– In Mathematica, PrimeQCertificate[23] yields { 23, 5, { 2, { 11, 2, { 2, { 5, 2, { 2 }}}}}}

The Succinctness of the Certificate

Lemma 58 The length of C(p) is at most quadratic at 5 log²₂ p.

• This claim holds when p = 2 or p = 3.

• In general, p − 1 has k ≤ log₂ p prime divisors q1 = 2, q2, . . . , q_k.

– Reason:

2^k ≤

k i=1

q_i ≤ p − 1.

• Note also that, as q1 = 2,

k

The Proof (continued)

• C(p) requires:

– 2 parentheses;

– 2k < 2 log₂ p separators (at most 2 log₂ p bits);

– r (at most log₂ p bits);

– q1 = 2 and its certificate 1 (at most 5 bits);

– q2, . . . , q_k (at most 2 log₂ p bits);^a – C(q2), . . . , C(qk).

aWhy?

The Proof (concluded)

• C(p) is succinct because, by induction,

| C(p) | ≤ 5 log₂ p + 5 + 5

k i=2

log²₂ q_i

≤ 5 log₂ p + 5 + 5

_k



i=2

log₂ q_i ²

≤ 5 log₂ p + 5 + 5 log²₂ p − 1

2 by inequality (6)

< 5 log₂ p + 5 + 5[ (log₂ p) − 1 ]²

= 5 log² p + 10 − 5 log p ≤ 5 log² p

Turning the Proof into an Algorithm

• How to turn the proof into a nondeterministic polynomial-time algorithm?

• First, guess a log₂ p-bit number r.

• Then guess up to log₂ p numbers q1, q2, . . . , q_k each containing at most log₂ p bits.

• Then recursively do the same thing for each of the q_i to form a certificate (5) on p. 480.

• Finally check if the two conditions of Theorem 56 (p.

476) hold throughout the tree.

aContributed by Mr. Kai-Yuan Hou (B99201038, R03922014) on

Euler’s

Totient or Phi Function

• Let

Φ(n) = { m : 1 ≤ m < n, gcd(m, n) = 1 }

be the set of all positive integers less than n that are prime to n.^b

– Φ(12) = { 1, 5, 7, 11 }.

• Define Euler’s function of n to be φ(n) = | Φ(n) |.

• φ(p) = p − 1 for prime p, and φ(1) = 1 by convention.

• Euler’s function is not expected to be easy to compute without knowing n’s factorization.

Leonhard Euler (1707–1783)

Three Properties of Euler’s Function

The inclusion-exclusion principle^b can be used to prove the following.

Lemma 59 If n = p^e₁¹p^e₂² · · · p^e_^ is the prime factorization of n, then

φ(n) = n

 i=1

1 − 1 p_i

 .

• For example, if n = pq, where p and q are distinct primes, then

φ(n) = pq 

1 − 1 p

1 − 1 q

= pq − p − q + 1.

aSee p. 224 of the textbook.

Three Properties of Euler’s Function (concluded)

Corollary 60 φ(mn) = φ(m) φ(n) if gcd(m, n) = 1.

Lemma 61 (Gauss) 

m|n φ(m) = n.

The Chinese Remainder Theorem

• Let n = n1n2 · · · nk, where n_i are pairwise relatively prime.

• For any integers a1, a2, . . . , ak, the set of simultaneous equations

x = a1 mod n1, x = a2 mod n2,

...

x = a_k mod n_k,

has a unique solution modulo n for the unknown x.

Fermat’s “Little” Theorem

Lemma 62 For all 0 < a < p, a^p−1 = 1 mod p.

• Recall Φ(p) = { 1, 2, . . . , p − 1 }.

• Consider aΦ(p) = { am mod p : m ∈ Φ(p) }.

• aΦ(p) = Φ(p).

– aΦ(p) ⊆ Φ(p) as a remainder must be between 1 and p − 1.

– Suppose am ≡ am^ mod p for m > m^, where m, m^ ∈ Φ(p).

– That means a(m − m^) = 0 mod p, and p divides a or

The Proof (concluded)

• Multiply all the numbers in Φ(p) to yield (p − 1)!.

• Multiply all the numbers in aΦ(p) to yield a^p−1(p − 1)!.

• As aΦ(p) = Φ(p), we have

a^p−1(p − 1)! ≡ (p − 1)! mod p.

• Finally, a^p−1 = 1 mod p because p  |(p − 1)!.

The Fermat-Euler Theorem

Corollary 63 For all a ∈ Φ(n), a^φ(n) = 1 mod n.

• The proof is similar to that of Lemma 62 (p. 493).

• Consider aΦ(n) = { am mod n : m ∈ Φ(n) }.

• aΦ(n) = Φ(n).

– aΦ(n) ⊆ Φ(n) as a remainder must be between 0 and n − 1 and relatively prime to n.

– Suppose am ≡ am^ mod n for m^ < m < n, where m, m^ ∈ Φ(n).

– That means a(m − m^) = 0 mod n, and n divides a or m − m^, which is impossible.

The Proof (concluded)

• Multiply all the numbers in Φ(n) to yield

m∈Φ(n) m.

• Multiply all the numbers in aΦ(n) to yield a^φ(n)

m∈Φ(n) m.

• As aΦ(n) = Φ(n),



m∈Φ(n)

m ≡ a^φ(n)

⎛

⎝ 

m∈Φ(n)

m

⎞

⎠ mod n.

• Finally, a^φ(n) = 1 mod n because n  |

m∈Φ(n) m.

aSome typographical errors corrected by Mr. Jung-Ying Chen (D95723006) on November 18, 2008.

An Example

• As 12 = 2² × 3,

φ(12) = 12 × 

1 − 1 2

1 − 1 3

= 4.

• In fact, Φ(12) = { 1, 5, 7, 11 }.

• For example,

5⁴ = 625 = 1 mod 12.