A Random Walk Algorithm for φ in CNF Form

The Markov Inequality

Lemma 62 Let x be a random variable taking nonnegative integer values. Then for any k > 0,

prob[x ≥ kE[ x ] ] ≤ 1/k.

• Let p_i denote the probability that x = i.

E[ x ] = 

i

ipi = 

i<kE[ x ]

ipi + 

i≥kE[ x ]

ipi

≥ 

i≥kE[ x ]

ip_i ≥ kE[ x ] 

i≥kE[ x ]

p_i

≥ kE[ x ] × prob[x ≥ kE[ x ]].

aAndrei Andreyevich Markov (1856–1922).

Andrei Andreyevich Markov (1856–1922)

fsat for k-sat Formulas (p. 484)

• Let φ(x1, x2, . . . , x_n) be a k-sat formula.

• If φ is satisfiable, then return a satisfying truth assignment.

• Otherwise, return “no.”

• We next propose a randomized algorithm for this problem.

A Random Walk Algorithm for φ in CNF Form

1: Start with an arbitrary truth assignment T ;

2: for i = 1, 2, . . . , r do

3: if T |= φ then

4: return “φ is satisfiable with T ”;

5: else

6: Let c be an unsatisfied clause in φ under T ; {All of its literals are false under T .}

7: Pick any x of these literals at random;

8: Modify T to make x true;

9: end if

10: end for

11: return “φ is unsatisfiable”;

3sat vs. 2sat Again

• Note that if φ is unsatisfiable, the algorithm will answer

“unsatisfiable.”

• The random walk algorithm needs expected exponential time for 3sat.

– In fact, it runs in expected O((1.333 · · · + )ⁿ) time with r = 3n,^a much better than O(2ⁿ).^b

• We will show immediately that it works well for 2sat.

• The state of the art as of 2006 is expected O(1.322ⁿ) time for 3sat and expected O(1.474ⁿ) time for 4sat.^c

aUse this setting per run of the algorithm.

bSch¨oning (1999).

cKwama and Tamaki (2004); Rolf (2006).

Random Walk Works for 2sat

Theorem 63 Suppose the random walk algorithm with r = 2n² is applied to any satisfiable 2sat problem with n variables. Then a satisfying truth assignment will be

discovered with probability at least 0.5.

• Let ˆT be a truth assignment such that ˆT |= φ.

• Assume our starting T differs from ˆT in i values.

– Their Hamming distance is i.

– Recall T is arbitrary.

aPapadimitriou (1991).

The Proof

• Let t(i) denote the expected number of repetitions of the flipping step^a until a satisfying truth assignment is

found.

• It can be shown that t(i) is finite.

• t(0) = 0 because it means that T = ˆT and hence T |= φ.

• If T = ˆT or any other satisfying truth assignment, then we need to flip the coin at least once.

• We flip a coin to pick among the 2 literals of a clause not satisfied by the present T .

• At least one of the 2 literals is true under ˆT because ˆT satisfies all clauses.

aThat is, Statement 7.

The Proof (continued)

• So we have at least 0.5 chance of moving closer to ˆT .

• Thus

t(i) ≤ t(i − 1) + t(i + 1)

2 + 1

for 0 < i < n.

– Inequality is used because, for example, T may differ from ˆT in both literals.

• It must also hold that

t(n) ≤ t(n − 1) + 1 because at i = n, we can only decrease i.

The Proof (continued)

• Now, put the necessary relations together:

t(0) = 0, (9)

t(i) ≤ t(i − 1) + t(i + 1)

2 + 1, 0 < i < n, (10)

t(n) ≤ t(n − 1) + 1. (11)

• Technically, this is a one-dimensional random walk with an absorbing barrier at i = 0 and a reflecting barrier at i = n (if we replace “≤” with “=”).^a

aThe proof in the textbook does exactly that. But a student pointed out difficulties with this proof technique on December 8, 2004. So our proof here uses the original inequalities.

The Proof (continued)

• Add up the relations for

2t(1), 2t(2), 2t(3), . . . , 2t(n − 1), t(n) to obtain^a 2t(1) + 2t(2) + · · · + 2t(n − 1) + t(n)

≤ t(0) + t(1) + 2t(2) + · · · + 2t(n − 2) + 2t(n − 1) + t(n) +2(n − 1) + 1.

• Simplify it to yield

t(1) ≤ 2n − 1. (12)

aAdding up the relations for t(1), t(2), t(3), . . . , t(n−1) will also work, thanks to Mr. Yen-Wu Ti (D91922010).

The Proof (continued)

• Add up the relations for 2t(2), 2t(3), . . . , 2t(n − 1), t(n) to obtain

2t(2) + · · · + 2t(n − 1) + t(n)

≤ t(1) + t(2) + 2t(3) + · · · + 2t(n − 2) + 2t(n − 1) + t(n) +2(n − 2) + 1.

• Simplify it to yield

t(2) ≤ t(1) + 2n − 3 ≤ 2n − 1 + 2n − 3 = 4n − 4 by Eq. (12) on p. 528.

The Proof (continued)

• Continuing the process, we shall obtain t(i) ≤ 2in − i².

• The worst upper bound happens when i = n, in which case

t(n) ≤ n².

• We conclude that

t(i) ≤ t(n) ≤ n² for 0 ≤ i ≤ n.

The Proof (concluded)

• So the expected number of steps is at most n².

• The algorithm picks r = 2n².

– This amounts to invoking the Markov inequality

(p. 519) with k = 2, resulting in a probability of 0.5.

• The proof does not yield a polynomial bound for 3sat.^a

aContributed by Mr. Cheng-Yu Lee (R95922035) on November 8, 2006.

Boosting the Performance

• We can pick r = 2mn² to have an error probability of

≤ 1 2m by Markov’s inequality.

• Alternatively, with the same running time, we can run the “r = 2n²” algorithm m times.

• The error probability is now reduced to

≤ 2^−m.

Primality Tests

• primes asks if a number N is a prime.

• The classic algorithm tests if k | N for k = 2, 3, . . . ,√ N .

• But it runs in Ω(2^{(log2 N)/2}) steps.

Primality Tests (concluded)

• Suppose N = P Q is a product of 2 distinct primes.

• The probability of success of the density attack (p. 468) is

≈ 2

√N when P ≈ Q.

• This probability is exponentially small in terms of the input length log₂ N.

The Fermat Test for Primality

Fermat’s “little” theorem (p. 471) suggests the following primality test for any given number N:

1: Pick a number a randomly from { 1, 2, . . . , N − 1 };

2: if a^N−1 ≡ 1 mod N then

3: return “N is composite”;

4: else

5: return “N is (probably) a prime”;

6: end if

The Fermat Test for Primality (concluded)

• Carmichael numbers are composite numbers that will pass the Fermat test for all a ∈ { 1, 2, . . . , N − 1 }.^a

– The Fermat test will return “N is a prime” for all Carmichael numbers N.

• Unfortunately, there are infinitely many Carmichael numbers.^b

• In fact, the number of Carmichael numbers less than N exceeds N^2/7 for N large enough.

• So the Fermat test is an incorrect algorithm for primes.

aCarmichael (1910). Lo (1994) mentions an investment strategy based on such numbers!

bAlford, Granville, and Pomerance (1992).

Square Roots Modulo a Prime

• Equation x² ≡ a mod p has at most two (distinct) roots by Lemma 59 (p. 476).

– The roots are called square roots.

– Numbers a with square roots and gcd(a, p) = 1 are called quadratic residues.

∗ They are

1² mod p, 2² mod p, . . . , (p − 1)² mod p.

• We shall show that a number either has two roots or has none, and testing which is the case is trivial.^a

aBut no efficient deterministic general-purpose square-root-extracting algorithms are known yet.

Euler’s Test

Lemma 64 (Euler) Let p be an odd prime and a = 0 mod p.

1. If

a^(p−1)/2 ≡ 1 mod p, then x² ≡ a mod p has two roots.

2. If

a^(p−1)/2 ≡ 1 mod p, then

a^(p−1)/2 ≡ −1 mod p and x² ≡ a mod p has no roots.

The Proof (continued)

• Let r be a primitive root of p.

• Fermat’s “little” theorem says r^p−1 ≡ 1 mod p, so r^(p−1)/2

is a square root of 1.

• In particular,

r^(p−1)/2 ≡ 1 or −1 mod p.

• But as r is a primitive root, r^(p−1)/2 ≡ 1 mod p.

• Hence r^(p−1)/2 ≡ −1 mod p.

The Proof (continued)

• Let a = r^k mod p for some k.

• Then

1 ≡ a^(p−1)/2 ≡ r^k(p−1)/2 ≡ 

r^(p−1)/2 _k

≡ (−1)^k mod p.

• So k must be even.

The Proof (continued)

• Suppose a = r^2j mod p for some 1 ≤ j ≤ (p − 1)/2.

• Then

a^(p−1)/2 ≡ r^j(p−1) ≡ 1 mod p.

• The two distinct roots of a are

r^j, −r^j(≡ r^j+(p−1)/2 mod p).

– If r^j ≡ −r^j mod p, then 2r^j ≡ 0 mod p, which implies r^j ≡ 0 mod p, a contradiction as r is a primitive root.

The Proof (continued)

• As 1 ≤ j ≤ (p − 1)/2, there are (p − 1)/2 such a’s.

• Each such a ≡ r^2j mod p has 2 distinct square roots.

• The square roots of all these a’s are distinct.

– The square roots of different a’s must be different.

• Hence the set of square roots is { 1, 2, . . . , p − 1 }.

• As a result,

a = r^2j mod p, 1 ≤ j ≤ (p − 1)/2, exhaust all the quadratic residues.

The Proof (concluded)

• Suppose a = r^2j+1 mod p now.

• Then it has no square roots because all the square roots have been taken.

• Finally,

a^(p−1)/2 ≡ 

r^(p−1)/2 _2j+1

≡ (−1)^2j+1 ≡ −1 mod p.

The Legendre Symbol^a and Quadratic Residuacity Test

• By Lemma 64 (p. 538),

a^(p−1)/2 mod p = ±1 for a ≡ 0 mod p.

• For odd prime p, define the Legendre symbol (a | p) as

(a | p) =

⎧⎪

⎪⎨

⎪⎪

⎩

0 if p | a,

1 if a is a quadratic residue modulo p,

−1 if a is a quadratic nonresidue modulo p.

• It is sometimes pronounced “a over p.”

aAndrien-Marie Legendre (1752–1833).

The Legendre Symbol and Quadratic Residuacity Test (concluded)

• Euler’s test (p. 538) implies

a^(p−1)/2 ≡ (a | p) mod p for any odd prime p and any integer a.

• Note that (ab | p) = (a | p)(b | p).

Gauss’s Lemma

Lemma 65 (Gauss) Let p and q be two distinct odd primes. Then (q | p) = (−1)^m, where m is the number of residues in R = { iq mod p : 1 ≤ i ≤ (p − 1)/2 } that are greater than (p − 1)/2.

• All residues in R are distinct.

– If iq = jq mod p, then p | (j − i) or p | q.

– But neither is possible.

• No two elements of R add up to p.

– If iq + jq ≡ 0 mod p, then p | (i + j) or p | q.

– But neither is possible.

The Proof (continued)

• Replace each of the m elements a ∈ R such that a > (p − 1)/2 by p − a.

– This is equivalent to performing −a mod p.

• Call the resulting set of residues R^.

• All numbers in R^ are at most (p − 1)/2.

• In fact, R^ = { 1, 2, . . . , (p − 1)/2 } (see illustration next page).

– Otherwise, two elements of R would add up to p,^a which has been shown to be impossible.

aBecause iq ≡ −jq mod p for some i = j.

5 1 2 3 4

6 5

1 2 3 4

6

p = 7 and q = 5.

The Proof (concluded)

• Alternatively, R^ = { ±iq mod p : 1 ≤ i ≤ (p − 1)/2 }, where exactly m of the elements have the minus sign.

• Take the product of all elements in the two representations of R^.

• So

[(p − 1)/2]! = (−1)^mq^(p−1)/2[(p − 1)/2]! mod p.

• Because gcd([(p − 1)/2]!, p) = 1, the above implies 1 = (−1)^mq^(p−1)/2 mod p.

Legendre’s Law of Quadratic Reciprocity

• Let p and q be two distinct odd primes.

• The next result says (p | q) and (q | p) are distinct if and only if both p and q are 3 mod 4.

Lemma 66 (Legendre (1785), Gauss)

(p | q)(q | p) = (−1)^{p−12 q−12} .

aFirst stated by Euler in 1751. Legendre (1785) did not give a cor- rect proof. Gauss proved the theorem when he was 19. He gave at least 8 different proofs during his life. The 152nd proof appeared in 1963. A computer-generated formal proof was given in Russinoff (1990).

As of 2008, there have been 4 such proofs. Wiedijk (2008), “the Law of Quadratic Reciprocity is the first nontrivial theorem that a student encounters in the mathematics curriculum.”

The Proof (continued)

• Sum the elements of R^ in the previous proof in mod2.

• On one hand, this is just _(p−1)/2

i=1 i mod 2.

• On the other hand, the sum equals mp +

(p−1)/2

i=1



iq − p

iq p



mod 2

= mp +

⎛

⎝q ^(p−1)/2

i=1

i − p

(p−1)/2

i=1

iq p

⎞⎠ mod 2.

– m of the iq mod p are replaced by p − iq mod p.

– But signs are irrelevant under mod2.

– m is as in Lemma 65 (p. 546).

The Proof (continued)

• Ignore odd multipliers to make the sum equal

m +

⎛

⎝^(p−1)/2

i=1

i −

(p−1)/2

i=1

iq p

⎞⎠ mod 2.

• Equate the above with _(p−1)/2

i=1 i modulo 2 and simplify to obtain

m ≡

(p−1)/2

i=1

iq p



mod 2.

The Proof (continued)

• _(p−1)/2

i=1 ^iq_p  is the number of integral points below the line

y = (q/p) x for 1 ≤ x ≤ (p − 1)/2.

• Gauss’s lemma (p. 546) says (q | p) = (−1)^m.

• Repeat the proof with p and q reversed.

• Then (p | q) = (−1)^m, where m^ is the number of integral points above the line y = (q/p) x for

1 ≤ y ≤ (q − 1)/2.

The Proof (concluded)

• As a result,

(p | q)(q | p) = (−1)^m+m.

• But m + m^ is the total number of integral points in the [1, ^p−1₂ ] × [1, ^q−1₂ ] rectangle, which is

p − 1 2

q − 1 2 .

Eisenstein’s Rectangle

(p,q)

(p - 1)/2 (q - 1)/2

Above, p = 11, q = 7, m = 7, m^ = 8.

The Jacobi Symbol

• The Legendre symbol only works for odd prime moduli.

• The Jacobi symbol (a | m) extends it to cases where m is not prime.

– a is sometimes called the numerator and m the denominator.

• Define (a | 1) = 1.

aCarl Jacobi (1804–1851).

The Jacobi Symbol (concluded)

• Let m = p¹p² · · · pk be the prime factorization of m.

• When m > 1 is odd and gcd(a, m) = 1, then

(a | m) =

k i=1

(a | p_i).

– Note that the Jacobi symbol equals ±1.

– It reduces to the Legendre symbol when m is a prime.

Properties of the Jacobi Symbol

The Jacobi symbol has the following properties when it is defined.

1. (ab | m) = (a | m)(b | m).

2. (a | m1m2) = (a | m1)(a | m2).

3. If a ≡ b mod m, then (a | m) = (b | m).

4. (−1 | m) = (−1)^(m−1)/2 (by Lemma 65 on p. 546).

5. (2 | m) = (−1)^(m2−1)/8.^a

6. If a and m are both odd, then (a | m)(m | a) = (−1)(a−1)(m−1)/4.

aBy Lemma 65 (p. 546) and some parity arguments.

Properties of the Jacobi Symbol (concluded)

• These properties allow us to calculate the Jacobi symbol without factorization.

– It will also yield the same result as Euler’s test (p.

538) when m is an odd prime.

• This situation is similar to the Euclidean algorithm.

• Note also that (a | m) = 1/(a | m) because (a | m) = ±1.^a

aContributed by Mr. Huang, Kuan-Lin (B96902079, R00922018) on December 6, 2011.

Calculation of (2200 | 999)

(2200| 999) = (202 | 999)

= (2 | 999)(101 | 999)

= (−1)^(9992−1)/8(101| 999)

= (−1)¹²⁴⁷⁵⁰(101| 999) = (101 | 999)

= (−1)(100)(998)/4(999| 101) = (−1)²⁴⁹⁵⁰(999| 101)

= (999| 101) = (90 | 101) = (−1)^(1012−1)/8(45| 101)

= (−1)¹²⁷⁵(45| 101) = −(45 | 101)

= −(−1)(44)(100)/4(101| 45) = −(101 | 45) = −(11 | 45)

= −(−1)^(10)(44)/4(45| 11) = −(45 | 11)

= −(1 | 11) = −1.

A Result Generalizing Proposition 10.3 in the Textbook

Theorem 67 The group of set Φ(n) under multiplication mod n has a primitive root if and only if n is either 1, 2, 4, p^k, or 2p^k for some nonnegative integer k and an odd prime p.

This result is essential in the proof of the next lemma.

The Jacobi Symbol and Primality Test

Lemma 68 If (M | N) ≡ M^(N−1)/2 mod N for all M ∈ Φ(N), then N is a prime. (Assume N is odd.)

• Assume N = mp, where p is an odd prime, gcd(m, p) = 1, and m > 1 (not necessarily prime).

• Let r ∈ Φ(p) such that (r | p) = −1.

• The Chinese remainder theorem says that there is an M ∈ Φ(N) such that

M = r mod p, M = 1 mod m.

aMr. Clement Hsiao (B4506061, R88526067) pointed out that the text- book’s proof for Lemma 11.8 is incorrect in January 1999 while he was a senior.

The Proof (continued)

• By the hypothesis,

M^(N−1)/2 = (M | N) = (M | p)(M | m) = −1 mod N.

• Hence

M^(N−1)/2 = −1 mod m.

• But because M = 1 mod m,

M^(N−1)/2 = 1 mod m, a contradiction.

The Proof (continued)

• Second, assume that N = p^a, where p is an odd prime and a ≥ 2.

• By Theorem 67 (p. 561), there exists a primitive root r modulo p^a.

• From the assumption, M^N−1 =

M^(N−1)/2 2

= (M|N)² = 1 mod N for all M ∈ Φ(N).

The Proof (continued)

• As r ∈ Φ(N) (prove it), we have

r^N−1 = 1 mod N.

• As r’s exponent modulo N = p^a is φ(N) = p^a−1(p − 1), p^a−1(p − 1) | (N − 1),

which implies that p | (N − 1).

• But this is impossible given that p | N.

The Proof (continued)

• Third, assume that N = mp^a, where p is an odd prime, gcd(m, p) = 1, m > 1 (not necessarily prime), and a is even.

• The proof mimics that of the second case.

• By Theorem 67 (p. 561), there exists a primitive root r modulo p^a.

• From the assumption, M^N−1 =

M^(N−1)/2 2

= (M|N)² = 1 mod N for all M ∈ Φ(N).

The Proof (continued)

• In particular,

M^N−1 = 1 mod p^a (13)

for all M ∈ Φ(N).

• The Chinese remainder theorem says that there is an M ∈ Φ(N) such that

M = r mod p^a, M = 1 mod m.

• Because M = r mod p^a and Eq. (13), r^N−1 = 1 mod p^a.

The Proof (concluded)

• As r’s exponent modulo N = p^a is φ(N) = p^a−1(p − 1), p^a−1(p − 1) | (N − 1),

which implies that p | (N − 1).

• But this is impossible given that p | N.

The Number of Witnesses to Compositeness

Theorem 69 (Solovay and Strassen (1977)) If N is an odd composite, then (M | N) ≡ M^(N−1)/2 mod N for at most half of M ∈ Φ(N).

• By Lemma 68 (p. 562) there is at least one a ∈ Φ(N) such that (a | N) ≡ a^(N−1)/2 mod N.

• Let B = { b1, b2, . . . , b_k } ⊆ Φ(N) be the set of all

distinct residues such that (bi | N) ≡ b^(N−1)/2_i mod N.

• Let aB = { abi mod N : i = 1, 2, . . . , k }.

• Clearly, aB ⊆ Φ(N), too.

The Proof (concluded)

• | aB | = k.

– ab_i ≡ ab_j mod N implies N | a(b_i − b_j), which is

impossible because gcd(a, N) = 1 and N > | bi − bj |.

• aB ∩ B = ∅ because

(abi)^(N−1)/2 ≡ a^(N−1)/2b^(N−1)/2_i ≡ (a | N)(bi | N) ≡ (abi | N).

• Combining the above two results, we know

| B |

φ(N) ≤ | B |

| B ∪ aB | = 0.5.

1: if N is even but N = 2 then

2: return “N is composite”;

3: else if N = 2 then

4: return “N is a prime”;

5: end if

6: Pick M ∈ { 2, 3, . . . , N − 1 } randomly;

7: if gcd(M, N ) > 1 then

8: return “N is composite”;

9: else

10: if (M | N ) ≡ M^(N−1)/2 mod N then

11: return “N is (probably) a prime”;

12: else

13: return “N is composite”;

14: end if

15: end if

Analysis

• The algorithm certainly runs in polynomial time.

• There are no false positives (for compositeness).

– When the algorithm says the number is composite, it is always correct.

Analysis (concluded)

• The probability of a false negative (again, for compositeness) is at most one half.

– Suppose the input is composite.

– By Theorem 69 (p. 569),

prob[ algorithm answers “no”| N is composite ] ≤ 0.5.

– Note that we are not referring to the probability that N is composite when the algorithm says “no.”

• So it is a Monte Carlo algorithm for compositeness.^a

aNot primes.

The Improved Density Attack for compositeness

All numbers < N

Witnesses to compositeness of

N via Jacobi Witnesses to

compositeness of N via common

factor

Randomized Complexity Classes; RP

• Let N be a polynomial-time precise NTM that runs in time p(n) and has 2 nondeterministic choices at each step.

• N is a polynomial Monte Carlo Turing machine for a language L if the following conditions hold:

– If x ∈ L, then at least half of the 2^p(n) computation paths of N on x halt with “yes” where n = | x |.

– If x ∈ L, then all computation paths halt with “no.”

• The class of all languages with polynomial Monte Carlo TMs is denoted RP (randomized polynomial time).^a

aAdleman and Manders (1977).

Comments on RP

• In analogy to Proposition 36 (p. 312), a “yes” instance of an RP problem has many certificates (witnesses).

• There are no false positives.

• If we associate nondeterministic steps with flipping fair coins, then we can phrase RP in the language of

probability.

– If x ∈ L, then N(x) halts with “yes” with probability at least 0.5 .

– If x ∈ L, then N(x) halts with “no.”

Comments on RP (concluded)

• The probability of false negatives is  ≤ 0.5.

• But any constant between 0 and 1 can replace 0.5.

– Repeat the algorithm k = −_log¹_{2 } times and answer

“no” only if all the runs answer “no.”

– The probability of false negatives becomes ^k ≤ 0.5.

Where RP Fits

• P ⊆ RP ⊆ NP.

– A deterministic TM is like a Monte Carlo TM except that all the coin flips are ignored.

– A Monte Carlo TM is an NTM with more demands on the number of accepting paths.

• compositeness ∈ RP;^a primes ∈ coRP;

primes ∈ RP.^b

– In fact, primes ∈ P.^c

• RP ∪ coRP is an alternative “plausible” notion of efficient computation.

aRabin (1976) and Solovay and Strassen (1977).

bAdleman and Huang (1987).

cAgrawal, Kayal, and Saxena (2002).