integer programming
• integer programming asks whether a system of linear inequalities with integer coefficients has an integer
solution.
• In contrast, linear programming asks whether a
system of linear inequalities with integer coefficients has a rational solution.
integer programming Is NP-Complete
a• set covering can be expressed by the inequalities Ax ≥ 1, n
i=1 xi ≤ B, 0 ≤ xi ≤ 1, where – xi is one if and only if Si is in the cover.
– A is the matrix whose columns are the bit vectors of the sets S1, S2, . . ..
– 1 is the vector of 1s.
– The operations in Ax are standard matrix operations.
• This shows integer programming is NP-hard.
• Many NP-complete problems can be expressed as an integer programming problem.
Christos Papadimitriou (1949–)
Easier or Harder?
a• Adding restrictions on the allowable problem instances will not make a problem harder.
– We are now solving a subset of problem instances or special cases.
– The independent set proof (p. 364) and the knapsack proof (p. 417): equally hard.
– circuit value to monotone circuit value (p. 317): equally hard.
– sat to 2sat (p. 344): easier.
aThanks to a lively class discussion on October 29, 2003.
Easier or Harder? (concluded)
• Adding restrictions on the allowable solutions (the solution space) may make a problem harder, equally hard, or easier.
• It is problem dependent.
– min cut to bisection width (p. 392): harder.
– linear programming to integer programming (p. 434): harder.
– sat to naesat (equally hard by p. 357) and max cut to max bisection (p. 390): equally hard.
– 3-coloring to 2-coloring (p. 401): easier.
coNP and Function Problems
coNP
• NP is the class of problems that have succinct certificates (recall Proposition 38 on p. 329).
• By definition, coNP is the class of problems whose complement is in NP.
• coNP is therefore the class of problems that have succinct disqualifications:
– A “no” instance of a problem in coNP possesses a short proof of its being a “no” instance.
– Only “no” instances have such proofs.
coNP (continued)
• Suppose L is a coNP problem.
• There exists a polynomial-time nondeterministic algorithm M such that:
– If x ∈ L, then M(x) = “yes” for all computation paths.
– If x ∈ L, then M(x) = “no” for some computation path.
• Note that if we swap “yes” and “no” of M, the new algorithm M decides ¯L ∈ NP in the classic sense (p.
103).
\HV [ ∉ /
\HV QR
\HV QR
\HV [ ∈ /
\HV
\HV
\HV
\HV
coNP (continued)
• So there are 3 major approaches to proving L ∈ coNP.
1. Prove ¯L ∈ NP.
2. Prove that only “no” instances possess short proofs.
3. Write an algorithm for it directly.
coNP (concluded)
• Clearly P ⊆ coNP.
• It is not known if
P = NP ∩ coNP.
– Contrast this with
R = RE ∩ coRE (see Proposition 14 on p. 169).
Some coNP Problems
• validity ∈ coNP.
– If φ is not valid, it can be disqualified very succinctly:
a truth assignment that does not satisfy it.
• sat complement ∈ coNP.
– sat complement is the complement of sat.
– The disqualification is a truth assignment that satisfies it.
• hamiltonian path complement ∈ coNP.
– The disqualification is a Hamiltonian path.
Some coNP Problems (concluded)
• optimal tsp (d) ∈ coNP.
– optimal tsp (d) asks if the optimal tour has a total distance of B, where B is an input.a
– The disqualification is a tour with a length < B.
aDefined by Mr. Che-Wei Chang (R95922093) on September 27, 2006.
A Nondeterministic Algorithm for sat complement (See also p. 113)
φ is a boolean formula with n variables.
1: for i = 1, 2, . . . , n do
2: Guess xi ∈ {0, 1}; {Nondeterministic choice.}
3: end for
4: {Verification:}
5: if φ(x1, x2, . . . , xn) = 1 then
6: “no”;
7: else
8: “yes”;
9: end if
Analysis
• The algorithm decides language {φ : φ is unsatisfiable}.
– The computation tree is a complete binary tree of depth n.
– Every computation path corresponds to a particular truth assignment out of 2n.
– φ is unsatisfiable if and only if every truth assignment falsifies φ.
– But every truth assignment falsifies φ if and only if every computation path results in “yes.”
An Alternative Characterization of coNP
Proposition 50 Let L ⊆ Σ∗ be a language. Then L ∈ coNP if and only if there is a polynomially decidable and
polynomially balanced relation R such that L = {x : ∀y (x, y) ∈ R}.
(As on p. 328, we assume | y | ≤ | x |k for some k.)
• ¯L = {x : ∃y (x, y) ∈ ¬R}.
• Because ¬R remains polynomially balanced, ¯L ∈ NP by Proposition 38 (p. 329).
• Hence L ∈ coNP by definition.
coNP-Completeness
Proposition 51 L is NP-complete if and only if its complement ¯L = Σ∗ − L is coNP-complete.
Proof (⇒; the ⇐ part is symmetric)
• Let ¯L be any coNP language.
• Hence L ∈ NP.
• Let R be the reduction from L to L.
• So x ∈ L if and only if R(x) ∈ L.
• By the law of transposition, x ∈ L if and only if R(x) ∈ L.
coNP Completeness (concluded)
• So x ∈ ¯L if and only if R(x) ∈ ¯L.
• The same R is a reduction from ¯L to ¯L.
• This shows ¯L is coNP-hard.
• But ¯L ∈ coNP.
• This shows ¯L is coNP-complete.
Some coNP-Complete Problems
• sat complement is coNP-complete.
• validity is coNP-complete.
– φ is valid if and only if ¬φ is not satisfiable.
– The reduction from sat complement to validity is hence easy.
• hamiltonian path complement is coNP-complete.
Possible Relations between P, NP, coNP
1. P = NP = coNP.
2. NP = coNP but P = NP.
3. NP = coNP and P = NP.
• This is the current “consensus.”a
aCarl Gauss (1777–1855), “I could easily lay down a multitude of such propositions, which one could neither prove nor dispose of.”
The Primality Problem
• An integer p is prime if p > 1 and all positive numbers other than 1 and p itself cannot divide it.
• primes asks if an integer N is a prime number.
• Dividing N by 2, 3, . . . ,√
N is not efficient.
– The length of N is only log N, but √
N = 20.5 log N. – It is an exponential-time algorithm.
• A polynomial-time algorithm for primes was not found until 2002 by Agrawal, Kayal, and Saxena!
• The running time is ˜O(log7.5 N).
1: if n = ab for some a, b > 1 then
2: return “composite”;
3: end if
4: for r = 2, 3, . . . , n − 1 do
5: if gcd(n, r) > 1 then
6: return “composite”;
7: end if
8: if r is a prime then
9: Let q be the largest prime factor of r − 1;
10: if q ≥ 4√r log n and n(r−1)/q = 1 mod r then
11: break; {Exit the for-loop.}
12: end if 13: end if
14: end for{r − 1 has a prime factor q ≥ 4√
r log n.}
15: for a = 1, 2, . . . , 2√
r log n do
16: if (x − a)n = (xn − a) mod (xr − 1) in Zn[x ] then
17: return “composite”;
18: end if 19: end for
The Primality Problem (concluded)
• Later, we will focus on efficient “randomized” algorithms for primes (used in Mathematica, e.g.).
• NP ∩ coNP is the class of problems that have succinct certificates and succinct disqualifications.
– Each “yes” instance has a succinct certificate.
– Each “no” instance has a succinct disqualification.
– No instances have both.
• We will see that primes ∈ NP ∩ coNP.
– In fact, primes ∈ P as mentioned earlier.
Primitive Roots in Finite Fields
Theorem 52 (Lucas and Lehmer (1927)) a A number p > 1 is a prime if and only if there is a number 1 < r < p such that
1. rp−1 = 1 mod p, and
2. r(p−1)/q = 1 mod p for all prime divisors q of p − 1.
• This r is called the primitive root or generator.
• We will prove the theorem later.b
aFran¸cois Edouard Anatole Lucas (1842–1891); Derrick Henry Lehmer (1905–1991).
bSee pp. 469ff.
Derrick Lehmer
a(1905–1991)
a
Pratt’s Theorem
Theorem 53 (Pratt (1975)) primes ∈ NP ∩ coNP.
• primes is in coNP because a succinct disqualification is a proper divisor.
– A proper divisor of a number n means n is not a prime.
• Now suppose p is a prime.
• p’s certificate includes the r in Theorem 52 (p. 457).
• Use recursive doubling to check if rp−1 = 1 mod p in time polynomial in the length of the input, log2 p.
– r, r2, r4, . . . mod p, a total of ∼ log p steps.
The Proof (concluded)
• We also need all prime divisors of p − 1: q1, q2, . . . , qk. – Whether r, q1, . . . , qk are easy to find is irrelevant.
– There may be multiple choices for r.
• Checking r(p−1)/qi = 1 mod p is also easy.
• Checking q1, q2, . . . , qk are all the divisors of p − 1 is easy.
• We still need certificates for the primality of the qi’s.
• The complete certificate is recursive and tree-like:
C(p) = (r; q1, C(q1), q2, C(q2), . . . , qk, C(qk)). (4)
• We next prove that C(p) is succinct.
The Succinctness of the Certificate
Lemma 54 The length of C(p) is at most quadratic at 5 log22 p.
• This claim holds when p = 2 or p = 3.
• In general, p − 1 has k ≤ log2 p prime divisors q1 = 2, q2, . . . , qk.
– Reason:
2k ≤
k i=1
qi ≤ p − 1.
• Note also that, as q1 = 2,
k
qi ≤ p − 1
2 . (5)
The Proof (continued)
• C(p) requires:
– 2 parentheses;
– 2k < 2 log2 p separators (at most 2 log2 p bits);
– r (at most log2 p bits);
– q1 = 2 and its certificate 1 (at most 5 bits);
– q2, . . . , qk (at most 2 log2 p bits);a – C(q2), . . . , C(qk).
aWhy?
The Proof (concluded)
• C(p) is succinct because, by induction,
|C(p)| ≤ 5 log2 p + 5 + 5
k i=2
log22 qi
≤ 5 log2 p + 5 + 5
k
i=2
log2 qi
2
≤ 5 log2 p + 5 + 5 log22 p − 1
2 by inequality (5)
< 5 log2 p + 5 + 5[ (log2 p) − 1 ]2
= 5 log22 p + 10 − 5 log2 p ≤ 5 log22 p for p ≥ 4.
A Certificate for 23
a• Note that 5 is a primitive root modulo 23 and 23 − 1 = 22 = 2 × 11.b
• So
C(23) = (5; 2, C(2), 11, C(11)).
• Note that 2 is a primitive root modulo 11 and 11 − 1 = 10 = 2 × 5.
• So
C(11) = (2; 2, C(2), 5, C(5)).
aThanks to a lively discussion on April 24, 2008.
bOther primitive roots are 7, 10, 11, 14, 15, 17, 19, 20, 21.
A Certificate for 23 (concluded)
• Note that 2 is a primitive root modulo 5 and 5 − 1 = 4 = 22.
• So
C(5) = (2; 2, C(2)).
• In summary,
C(23) = (5; 2, C(2), 11, (2; 2, C(2), 5, (2; 2, C(2)))).
– In Mathematica, PrimeQCertificate[23] yields {23, 5, {2, {11, 2, {2, {5, 2, {2}}}}}}
Turning the Proof into an Algorithm
a• How to turn the proof into a polynomial-time nondeterministic algorithm?
• First, guess a log2 p-bit number r.
• Then guess up to log2 p log2 p-bit numbers q1, q2, . . . , qk.
• Then recursively do the same thing for each of the qi to form a certificate (4) on p. 460.
• Finally check if the two conditions of Theorem 52 (p.
457) hold throughout the tree.
aContributed by Mr. Kai-Yuan Hou (B99201038, R03922014) on November 24, 2015.
Basic Modular Arithmetics
a• Let m, n ∈ Z+.
• m | n means m divides n; m is n’s divisor.
• We call the numbers 0, 1, . . . , n − 1 the residue modulo n.
• The greatest common divisor of m and n is denoted gcd(m, n).
• The r in Theorem 52 (p. 457) is a primitive root of p.
• We now prove the existence of primitive roots and then Theorem 52 (p. 457).
a
Basic Modular Arithmetics (concluded)
• We use
a ≡ b mod n if n | (a − b).
– So 25 ≡ 38 mod 13.
• We use
a = b mod n
if b is the remainder of a divided by n.
– So 25 = 12 mod 13.
Euler’s
aTotient or Phi Function
• Let
Φ(n) = {m : 1 ≤ m < n, gcd(m, n) = 1}
be the set of all positive integers less than n that are prime to n.b
– Φ(12) = {1, 5, 7, 11}.
• Define Euler’s function of n to be φ(n) = | Φ(n) |.
• φ(p) = p − 1 for prime p, and φ(1) = 1 by convention.
• Euler’s function is not expected to be easy to compute without knowing n’s factorization.
aLeonhard Euler (1707–1783).
Two Properties of Euler’s Function
The inclusion-exclusion principlea can be used to prove the following.
Lemma 55 φ(n) = n
p|n(1 − 1p).
• If n = pe11pe22 · · · pe is the prime factorization of n, then φ(n) = n
i=1
1 − 1 pi
.
Corollary 56 φ(mn) = φ(m) φ(n) if gcd(m, n) = 1.
aConsult any textbooks on discrete mathematics.
A Key Lemma
Lemma 57
m|n φ(m) = n.
• Let n =
i=1 pkii be the prime factorization of n and consider
i=1
[φ(1) + φ(pi) + · · · + φ(pkii) ]. (6)
• Equation (6) equals n because φ(pki ) = pki − pk−1i by Lemma 55 (p. 471) so φ(1) + φ(pi) + · · · + φ(pkii) = pkii.
• Expand Eq. (6) to yield
n =
k≤k ,...,k≤k
i=1
φ(pkii).
The Proof (concluded)
• By Corollary 56 (p. 471),
i=1
φ(pkii) = φ
i=1
pkii
.
• So Eq. (6) becomes
n =
k1≤k1,...,k≤k
φ
i=1
pkii
.
• Each
i=1 pkii is a unique divisor of n =
i=1 pkii.
• Equation (6) becomes
φ(m).
Leonhard Euler (1707–1783)
The Density Attack for primes
Witnesses to compositeness
of n
All numbers < n
The Density Attack for primes
1: Pick k ∈ {1, . . . , n} randomly;
2: if k | n and k = 1 and k = n then
3: return “n is composite”;
4: else
5: return “n is (probably) a prime”;
6: end if
The Density Attack for primes (continued)
• It works, but does it work well?
• The ratio of numbers ≤ n relatively prime to n (the white ring) is
φ(n) n .
• When n = pq, where p and q are distinct primes, φ(n)
n = pq − p − q + 1
pq > 1 − 1
q − 1 p.
The Density Attack for primes (concluded)
• So the ratio of numbers ≤ n not relatively prime to n (the grey area) is < (1/q) + (1/p).
– The “density attack” has probability about 2/√
n of factoring n = pq when p ∼ q = O(√
n ).
– The “density attack” to factor n = pq hence takes Ω(√
n) steps on average when p ∼ q = O(√ n ).
– This running time is exponential: Ω(20.5 log2n).
The Chinese Remainder Theorem
• Let n = n1n2 · · · nk, where ni are pairwise relatively prime.
• For any integers a1, a2, . . . , ak, the set of simultaneous equations
x = a1 mod n1, x = a2 mod n2,
...
x = ak mod nk,
has a unique solution modulo n for the unknown x.
Fermat’s “Little” Theorem
aLemma 58 For all 0 < a < p, ap−1 = 1 mod p.
• Recall Φ(p) = {1, 2, . . . , p − 1}.
• Consider aΦ(p) = {am mod p : m ∈ Φ(p)}.
• aΦ(p) = Φ(p).
– aΦ(p) ⊆ Φ(p) as a remainder must be between 1 and p − 1.
– Suppose am ≡ am mod p for m > m, where m, m ∈ Φ(p).
– That means a(m − m) = 0 mod p, and p divides a or m − m, which is impossible.
The Proof (concluded)
• Multiply all the numbers in Φ(p) to yield (p − 1)!.
• Multiply all the numbers in aΦ(p) to yield ap−1(p − 1)!.
• As aΦ(p) = Φ(p), we have
ap−1(p − 1)! ≡ (p − 1)! mod p.
• Finally, ap−1 = 1 mod p because p |(p − 1)!.
The Fermat-Euler Theorem
aCorollary 59 For all a ∈ Φ(n), aφ(n) = 1 mod n.
• The proof is similar to that of Lemma 58 (p. 480).
• Consider aΦ(n) = {am mod n : m ∈ Φ(n)}.
• aΦ(n) = Φ(n).
– aΦ(n) ⊆ Φ(n) as a remainder must be between 0 and n − 1 and relatively prime to n.
– Suppose am ≡ am mod n for m < m < n, where m, m ∈ Φ(n).
– That means a(m − m) = 0 mod n, and n divides a or m − m, which is impossible.
aProof by Mr. Wei-Cheng Cheng (R93922108, D95922011) on Novem-
The Proof (concluded)
a• Multiply all the numbers in Φ(n) to yield
m∈Φ(n) m.
• Multiply all the numbers in aΦ(n) to yield aφ(n)
m∈Φ(n) m.
• As aΦ(n) = Φ(n),
m∈Φ(n)
m ≡ aφ(n)
⎛
⎝
m∈Φ(n)
m
⎞
⎠ mod n.
• Finally, aφ(n) = 1 mod n because n |
m∈Φ(n) m.
aSome typographical errors corrected by Mr. Jung-Ying Chen (D95723006) on November 18, 2008.
An Example
• As 12 = 22 × 3,
φ(12) = 12 ×
1 − 1 2
1 − 1 3
= 4.
• In fact, Φ(12) = {1, 5, 7, 11}.
• For example,
54 = 625 = 1 mod 12.
Exponents
• The exponent of m ∈ Φ(p) is the least k ∈ Z+ such that mk = 1 mod p.
• Every residue s ∈ Φ(p) has an exponent.
– 1, s, s2, s3, . . . eventually repeats itself modulo p, say si ≡ sj mod p, which means sj−i = 1 mod p.
• If the exponent of m is k and m = 1 mod p, then k | .
– Otherwise, = qk + a for 0 < a < k, and
m = mqk+a ≡ ma ≡ 1 mod p, a contradiction.
Lemma 60 Any nonzero polynomial of degree k has at most k distinct roots modulo p.