Monte Carlo Algorithms
a• The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that
– If the algorithm finds that a matching exists, it is always correct (no false positives).
– If the algorithm answers in the negative, then it may make an error (false negative).
aMetropolis and Ulam (1949).
Monte Carlo Algorithms (concluded)
• The algorithm makes a false negative with probability
≤ 0.5.
– Note this probability refers to
prob[ algorithm answers “no” | G has a perfect matching ] not
prob[ G has a perfect matching | algorithm answers “no” ].
• This probability is not over the space of all graphs or determinants, but over the algorithm’s own coin flips.
– It holds for any bipartite graph.
False Positives and False Negatives in Human Behavior?
a• “[Men] tend to misinterpret innocent friendliness as a sign that women are [· · · ] interested in them.”
– A false positive.
• “[Women] tend to undervalue signs that a man is interested in a committed relationship.”
– A false negative.
a“Don’t misunderestimate yourself.” The Economist, 2006.
The Markov Inequality
aLemma 64 Let x be a random variable taking nonnegative integer values. Then for any k > 0,
prob[x ≥ kE[ x ]] ≤ 1/k.
• Let pi denote the probability that x = i.
E[ x ] = X
i
ipi
= X
i<kE[ x ]
ipi + X
i≥kE[ x ]
ipi
≥ kE[ x ] × prob[x ≥ kE[ x ]].
aAndrei Andreyevich Markov (1856–1922).
Andrei Andreyevich Markov (1856–1922)
An Application of Markov’s Inequality
• Algorithm C runs in expected time T (n) and always gives the right answer.
• Consider an algorithm that runs C for time kT (n) and rejects the input if C does not stop within the time bound.
• By Markov’s inequality, this new algorithm runs in time kT (n) and gives the wrong answer with probability
≤ 1/k.
• By running this algorithm m times, we reduce the error probability to ≤ k−m.
An Application of Markov’s Inequality (concluded)
• Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound.
• By Markov’s inequality, this new algorithm gives the wrong answer with probability ≤ 1/(mk).
• This is a far cry from the previous algorithm’s error probability of ≤ k−m.
fsat for k-sat Formulas (p. 427)
• Let φ(x1, x2, . . . , xn) be a k-sat formula.
• If φ is satisfiable, then return a satisfying truth assignment.
• Otherwise, return “no.”
• We next propose a randomized algorithm for this problem.
A Random Walk Algorithm for φ in CNF Form
1: Start with an arbitrary truth assignment T ;
2: for i = 1, 2, . . . , r do
3: if T |= φ then
4: return “φ is satisfiable with T ”;
5: else
6: Let c be an unsatisfiable clause in φ under T ; {All of its literals are false under T .}
7: Pick any x of these literals at random;
8: Modify T to make x true;
9: end if
10: end for
11: return “φ is unsatisfiable”;
3sat vs. 2sat Again
• Note that if φ is unsatisfiable, the algorithm will not refute it.
• The random walk algorithm needs expected exponential time for 3sat.
– In fact, it runs in expected O((1.333 · · · + ²)n) time with r = 3n,a much better than O(2n).b
• We will show immediately that it works well for 2sat.
• The state of the art as of 2006 is expected O(1.322n) time for 3sat and expected O(1.474n) time for 4sat.c
aUse this setting per run of the algorithm.
bSch¨oning (1999).
Random Walk Works for 2sat
aTheorem 65 Suppose the random walk algorithm with r = 2n2 is applied to any satisfiable 2sat problem with n variables. Then a satisfying truth assignment will be
discovered with probability at least 0.5.
• Let ˆT be a truth assignment such that ˆT |= φ.
• Let t(i) denote the expected number of repetitions of the flipping step until a satisfying truth assignment is found if our starting T differs from ˆT in i values.
– Their Hamming distance is i.
– Recall T is arbitrary.
aPapadimitriou (1991).
The Proof
• It can be shown that t(i) is finite.
• t(0) = 0 because it means that T = ˆT and hence T |= φ.
• If T 6= ˆT or T is not equal to any other satisfying truth assignment, then we need to flip at least once.
• We flip to pick among the 2 literals of a clause not satisfied by the present T .
• At least one of the 2 literals is true under ˆT because ˆT satisfies all clauses.
• So we have at least 0.5 chance of moving closer to ˆT .
The Proof (continued)
• Thus
t(i) ≤ t(i − 1) + t(i + 1)
2 + 1
for 0 < i < n.
– Inequality is used because, for example, T may differ from ˆT in both literals.
• It must also hold that
t(n) ≤ t(n − 1) + 1 because at i = n, we can only decrease i.
The Proof (continued)
• As we are only interested in upper bounds, we solve x(0) = 0
x(n) = x(n − 1) + 1
x(i) = x(i − 1) + x(i + 1)
2 + 1, 0 < i < n
• This is one-dimensional random walk with a reflecting and an absorbing barrier.
The Proof (continued)
• Add the equations up to obtain
x(1) + x(2) + · · · + x(n)
= x(0) + x(1) + 2x(2) + · · · + 2x(n − 2) + x(n − 1) + x(n) 2
+n + x(n − 1).
• Simplify to yield
x(1) + x(n) − x(n − 1)
2 = n.
• As x(n) − x(n − 1) = 1, we have x(1) = 2n − 1.
The Proof (continued)
• Iteratively, we obtain
x(2) = 4n − 4, ...
x(i) = 2in − i2.
• The worst case happens when i = n, in which case x(n) = n2.
The Proof (concluded)
• We therefore reach the conclusion that t(i) ≤ x(i) ≤ x(n) = n2.
• So the expected number of steps is at most n2.
• The algorithm picks a running time 2n2.
• This amounts to invoking the Markov inequality (p. 462) with k = 2, with the consequence of having a probability of 0.5.
• The proof does not yield a polynomial bound for 3sat.a
aContributed by Mr. Cheng-Yu Lee (R95922035) on November 8, 2006.
Boosting the Performance
• We can pick r = 2mn2 to have an error probability of
≤ (2m)−1 by Markov’s inequality.
• Alternatively, with the same running time, we can run the “r = 2n2” algorithm m times.
• But the error probability is reduced to ≤ 2−m!
Primality Tests
• primes asks if a number N is a prime.
• The classic algorithm tests if k | N for k = 2, 3, . . . ,√ N .
• But it runs in Ω(2n/2) steps, where n = | N | = log2 N .
The Density Attack for primes
1: Pick k ∈ {2, . . . , N − 1} randomly; {Assume N > 2.}
2: if k | N then
3: return “N is composite”;
4: else
5: return “N is a prime”;
6: end if
Analysis
a• Suppose N = P Q, a product of 2 primes.
• The probability of success is
< 1 − φ(N )
N = 1 − (P − 1)(Q − 1)
P Q = P + Q − 1 P Q .
• In the case where P ≈ Q, this probability becomes
< 1
P + 1
Q ≈ 2
√N .
• This probability is exponentially small.
aSee also p. 409.
The Fermat Test for Primality
Fermat’s “little” theorem on p. 411 suggests the following primality test for any given number p:
1: Pick a number a randomly from {1, 2, . . . , N − 1};
2: if aN −1 6= 1 mod N then
3: return “N is composite”;
4: else
5: return “N is a prime”;
6: end if
The Fermat Test for Primality (concluded)
• Unfortunately, there are composite numbers called Carmichael numbers that will pass the Fermat test for all a ∈ {1, 2, . . . , N − 1}.a
• There are infinitely many Carmichael numbers.b
• In fact, the number of Carmichael numbers less than n exceeds n2/7 for n large enough.
aCarmichael (1910).
bAlford, Granville, and Pomerance (1992).
Square Roots Modulo a Prime
• Equation x2 = a mod p has at most two (distinct) roots by Lemma 59 (p. 416).
– The roots are called square roots.
– Numbers a with square roots and gcd(a, p) = 1 are called quadratic residues.
∗ They are 12 mod p, 22 mod p, . . . , (p − 1)2 mod p.
• We shall show that a number either has two roots or has none, and testing which one is true is trivial.
• There are no known efficient deterministic algorithms to find the roots, however.
Euler’s Test
Lemma 66 (Euler) Let p be an odd prime and a 6= 0 mod p.
1. If a(p−1)/2 = 1 mod p, then x2 = a mod p has two roots.
2. If a(p−1)/2 6= 1 mod p, then a(p−1)/2 = −1 mod p and x2 = a mod p has no roots.
• Let r be a primitive root of p.
• By Fermat’s “little” theorem, r(p−1)/2 is a square root of 1, so r(p−1)/2 = 1 mod p or r(p−1)/2 = −1 mod p.
• But as r is a primitive root, r(p−1)/2 6= 1 mod p.
• Hence r(p−1)/2 = −1 mod p.
The Proof (continued)
• Let a = rk mod p for some k.
• Then
1 = a(p−1)/2 = rk(p−1)/2 = h
r(p−1)/2 ik
= (−1)k mod p.
• So k must be even.
• Suppose a = r2j for some 1 ≤ j ≤ (p − 1)/2.
• Then a(p−1)/2 = rj(p−1) = 1 mod p and its two distinct roots are rj, −rj(= rj+(p−1)/2 mod p).
– If rj = −rj mod p, then 2rj = 0 mod p, which implies rj = 0 mod p, a contradiction.
The Proof (continued)
• As 1 ≤ j ≤ (p − 1)/2, there are (p − 1)/2 such a’s.
• Each such a has 2 distinct square roots.
• The square roots of all the a’s are distinct.
– The square roots of different a’s must be different.
• Hence the set of square roots is {1, 2, . . . , p − 1}.
– Because there are (p − 1)/2 such a’s and each a has two square roots.
• As a result, a = r2j, 1 ≤ j ≤ (p − 1)/2, exhaust all the quadratic residues.
The Proof (concluded)
• If a = r2j+1, then it has no roots because all the square roots have been taken.
• Now,
a(p−1)/2 = h
r(p−1)/2
i2j+1
= (−1)2j+1 = −1 mod p.
The Legendre Symbola and Quadratic Residuacity Test
• By Lemma 66 (p. 483) a(p−1)/2 mod p = ±1 for a 6= 0 mod p.
• For odd prime p, define the Legendre symbol (a | p) as
(a | p) =
0 if p | a,
1 if a is a quadratic residue modulo p,
−1 if a is a quadratic nonresidue modulo p.
• Euler’s test implies a(p−1)/2 = (a | p) mod p for any odd prime p and any integer a.
• Note that (ab|p) = (a|p)(b|p).
aAndrien-Marie Legendre (1752–1833).
Gauss’s Lemma
Lemma 67 (Gauss) Let p and q be two odd primes. Then (q|p) = (−1)m, where m is the number of residues in
R = { iq mod p : 1 ≤ i ≤ (p − 1)/2 } that are greater than (p − 1)/2.
• All residues in R are distinct.
– If iq = jq mod p, then p|(j − i) q or p|q.
• No two elements of R add up to p.
– If iq + jq = 0 mod p, then p|(i + j) or p|q.
– But neither is possible.
The Proof (continued)
• Consider the set R0 of residues that result from R if we replace each of the m elements a ∈ R such that
a > (p − 1)/2 by p − a.
– This is equivalent to performing −a mod p.
• All residues in R0 are now at most (p − 1)/2.
• In fact, R0 = {1, 2, . . . , (p − 1)/2} (see illustration next page).
– Otherwise, two elements of R would add up to p, which has been shown to be impossible.
5 1 2 3 4
6 5
1 2 3 4
6
p = 7 and q = 5.
The Proof (concluded)
• Alternatively, R0 = {±iq mod p : 1 ≤ i ≤ (p − 1)/2}, where exactly m of the elements have the minus sign.
• Take the product of all elements in the two representations of R0.
• So [(p − 1)/2]! = (−1)mq(p−1)/2[(p − 1)/2]! mod p.
• Because gcd([(p − 1)/2]!, p) = 1, the above implies 1 = (−1)mq(p−1)/2 mod p.
Legendre’s Law of Quadratic Reciprocity
a• Let p and q be two odd primes.
• The next result says their Legendre symbols are distinct if and only if both numbers are 3 mod 4.
Lemma 68 (Legendre (1785), Gauss)
(p|q)(q|p) = (−1)p−12 q−12 .
aFirst stated by Euler in 1751. Legendre (1785) did not give a correct proof. Gauss proved the theorem when he was 19. He gave at least 6 different proofs during his life. The 152nd proof appeared in 1963.
The Proof (continued)
• Sum the elements of R0 in the previous proof in mod2.
• On one hand, this is just P(p−1)/2
i=1 i mod 2.
• On the other hand, the sum equals
(p−1)/2X
i=1
µ
qi − p
¹iq p
º¶
+ mp mod 2
=
q
(p−1)/2X
i=1
i − p
(p−1)/2X
i=1
¹iq p
º
+ mp mod 2.
– Signs are irrelevant under mod2.
– m is as in Lemma 67 (p. 488).
The Proof (continued)
• Ignore odd multipliers to make the sum equal
(p−1)/2X
i=1
i −
(p−1)/2X
i=1
¹iq p
º
+ m mod 2.
• Equate the above with P(p−1)/2
i=1 i mod 2 to obtain
m =
(p−1)/2X
i=1
¹iq p
º
mod 2.
The Proof (concluded)
• P(p−1)/2
i=1 biqp c is the number of integral points under the line y = (q/p) x for 1 ≤ x ≤ (p − 1)/2.
• Gauss’s lemma (p. 488) says (q|p) = (−1)m.
• Repeat the proof with p and q reversed.
• So (p|q) = (−1)m0, where m0 is the number of integral points above the line y = (q/p) x for 1 ≤ y ≤ (q − 1)/2.
• As a result, (p|q)(q|p) = (−1)m+m0.
• But m + m0 is the total number of integral points in the
p−1
2 × q−12 rectangle, which is p−12 q−12 .
Eisenstein’s Rectangle
(p,q)
(p - 1)/2 (q - 1)/2
p = 11 and q = 7.
The Jacobi Symbol
a• The Legendre symbol only works for odd prime moduli.
• The Jacobi symbol (a | m) extends it to cases where m is not prime.
• Let m = p1p2 · · · pk be the prime factorization of m.
• When m > 1 is odd and gcd(a, m) = 1, then
(a|m) = Yk i=1
(a | pi).
– Note that the Jacobi symbol equals ±1.
– It reduces to the Legendre symbol when m is a prime.
• Define (a | 1) = 1.
Properties of the Jacobi Symbol
The Jacobi symbol has the following properties, for arguments for which it is defined.
1. (ab | m) = (a | m)(b | m).
2. (a | m1m2) = (a | m1)(a | m2).
3. If a = b mod m, then (a | m) = (b | m).
4. (−1 | m) = (−1)(m−1)/2 (by Lemma 67 on p. 488).
5. (2 | m) = (−1)(m2−1)/8.a
6. If a and m are both odd, then (a | m)(m | a) = (−1)(a−1)(m−1)/4.
Calculation of (2200|999)
Similar to the Euclidean algorithm and does not require factorization.
(202|999) = (−1)(9992−1)/8(101|999)
= (−1)124750(101|999) = (101|999)
= (−1)(100)(998)/4(999|101) = (−1)24950(999|101)
= (999|101) = (90|101) = (−1)(1012−1)/8(45|101)
= (−1)1275(45|101) = −(45|101)
= −(−1)(44)(100)/4(101|45) = −(101|45) = −(11|45)
= −(−1)(10)(44)/4(45|11) = −(45|11)
= −(1|11) = −1.
A Result Generalizing Proposition 10.3 in the Textbook
Theorem 69 The group of set Φ(n) under multiplication mod n has a primitive root if and only if n is either 1, 2, 4, pk, or 2pk for some nonnegative integer k and and odd
prime p.
This result is essential in the proof of the next lemma.
The Jacobi Symbol and Primality Test
aLemma 70 If (M |N ) = M(N −1)/2 mod N for all M ∈ Φ(N ), then N is prime. (Assume N is odd.)
• Assume N = mp, where p is an odd prime, gcd(m, p) = 1, and m > 1 (not necessarily prime).
• Let r ∈ Φ(p) such that (r | p) = −1.
• The Chinese remainder theorem says that there is an M ∈ Φ(N ) such that
M = r mod p, M = 1 mod m.
aMr. Clement Hsiao (R88526067) pointed out that the textbook’s proof for Lemma 11.8 is incorrect while he was a senior in January 1999.
The Proof (continued)
• By the hypothesis,
M(N −1)/2 = (M | N ) = (M | p)(M | m) = −1 mod N.
• Hence
M(N −1)/2 = −1 mod m.
• But because M = 1 mod m,
M(N −1)/2 = 1 mod m, a contradiction.
The Proof (continued)
• Second, assume that N = pa, where p is an odd prime and a ≥ 2.
• By Theorem 69 (p. 500), there exists a primitive root r modulo pa.
• From the assumption, MN −1 =
h
M(N −1)/2 i2
= (M |N )2 = 1 mod N for all M ∈ Φ(N ).
The Proof (continued)
• As r ∈ Φ(N ) (prove it), we have
rN −1 = 1 mod N.
• As r’s exponent modulo N = pa is φ(N ) = pa−1(p − 1), pa−1(p − 1) | N − 1,
which implies that p | N − 1.
• But this is impossible given that p | N .
The Proof (continued)
• Third, assume that N = mpa, where p is an odd prime, gcd(m, p) = 1, m > 1 (not necessarily prime), and a is even.
• The proof mimics that of the second case.
• By Theorem 69 (p. 500), there exists a primitive root r modulo pa.
• From the assumption, MN −1 =
h
M(N −1)/2 i2
= (M |N )2 = 1 mod N for all M ∈ Φ(N ).
The Proof (continued)
• In particular,
MN −1 = 1 mod pa (7)
for all M ∈ Φ(N ).
• The Chinese remainder theorem says that there is an M ∈ Φ(N ) such that
M = r mod pa, M = 1 mod m.
• Because M = r mod pa and Eq. (7), rN −1 = 1 mod pa.
The Proof (concluded)
• As r’s exponent modulo N = pa is φ(N ) = pa−1(p − 1), pa−1(p − 1) | N − 1,
which implies that p | N − 1.
• But this is impossible given that p | N .