*Randomized Computation*

I know that half my advertising works, I just don’t know which half.

— John Wanamaker

I know that half my advertising is a waste of money, I just don’t know which half!

— McGraw-Hill ad.

### Randomized Algorithms

^{a}

*•* Randomized algorithms flip unbiased coins.

*•* There are important problems for which there are no
*known efficient deterministic algorithms but for which*
very efficient randomized algorithms exist.

– Extraction of square roots, for instance.

*•* *There are problems where randomization is necessary.*

– Secure protocols.

*•* Randomized version can be more efficient.

– Parallel algorithm for maximal independent set.

aRabin (1976); Solovay and Strassen (1977).

### “Four Most Important Randomized Algorithms”

^{a}

1. Primality testing.^{b}

2. Graph connectivity using random walks.^{c}
3. Polynomial identity testing.^{d}

4. Algorithms for approximate counting.^{e}

aTrevisan (2006).

bRabin (1976); Solovay and Strassen (1977).

cAleliunas, Karp, Lipton, Lov´asz, and Rackoff (1979).

dSchwartz (1980); Zippel (1979).

eSinclair and Jerrum (1989).

### Bipartite Perfect Matching

*•* *We are given a bipartite graph G = (U, V, E).*

– *U = {u*_{1}*, u*_{2}*, . . . , u*_{n}*}.*

– *V = {v*_{1}*, v*_{2}*, . . . , v*_{n}*}.*

– *E ⊆ U × V .*

*•* We are asked if there is a perfect matching.

– *A permutation π of {1, 2, . . . , n} such that*
*(u*_{i}*, v*_{π(i)}*) ∈ E*

*for all u*_{i}*∈ U .*

### A Perfect Matching

X_{}

X_{}

X_{}

X_{}

X_{}

Y_{}

Y_{}

Y_{}

Y_{}

Y_{}

### Symbolic Determinants

*•* *Given a bipartite graph G, construct the n × n matrix*
*A*^{G}*whose (i, j)th entry A*^{G}_{ij}*is a variable x** _{ij}* if

*(u*_{i}*, v*_{j}*) ∈ E and zero otherwise.*

*•* *The determinant of A** ^{G}* is

*det(A*

*) = X*

^{G}*π*

*sgn(π)*
Y*n*
*i=1*

*A*^{G}_{i,π(i)}*.* (5)
– *π ranges over all permutations of n elements.*

– *sgn(π) is 1 if π is the product of an even number of*
*transpositions and −1 otherwise.*

– *Equivalently, sgn(π) = 1 if the number of (i, j)s such*
*that i < j) and π(i) > π(j) is even.*^{a}

### Determinant and Bipartite Perfect Matching

*•* In P

*π* *sgn(π)*Q_{n}

*i=1* *A*^{G}* _{i,π(i)}*, note the following:

– Each summand corresponds to a possible prefect
*matching π.*

– *As all variables appear only once, all of these*
summands are different monomials and will not
cancel.

*•* It is essentially an exhaustive enumeration.

Proposition 58 (Edmonds (1967)) *G has a perfect*
*matching if and only if det(A*^{G}*) is not identically zero.*

### A Perfect Matching in a Bipartite Graph

X_{}

X_{}

X_{}

X_{}

X_{}

Y_{}

Y_{}

Y_{}

Y_{}

Y_{}

### The Perfect Matching in the Determinant

*•* The matrix is

*A** ^{G}* =

0 0 *x*_{13} *x*_{14} 0

0 *x*_{22} 0 0 0

*x*_{31} 0 0 0 *x*_{35}

*x*_{41} 0 *x*_{43} *x*_{44} 0

*x*_{51} 0 0 0 *x*_{55}

*.*

*•* *det(A*^{G}*) = −x*_{14}*x*_{22}*x*_{35}*x*_{43}*x*_{51} *+ x*_{13}*x*_{22}*x*_{35}*x*_{44}*x*_{51} +
*x*_{14}*x*_{22}*x*_{31}*x*_{43}*x*_{55} *− x*_{13}*x*_{22}*x*_{31}*x*_{44}*x*_{55}, each denoting a
perfect matching.

### How To Test If a Polynomial Is Identically Zero?

*•* *det(A*^{G}*) is a polynomial in n*^{2} variables.

*•* *There are exponentially many terms in det(A** ^{G}*).

*•* Expanding the determinant polynomial is not feasible.

– Too many terms.

*•* *Observation: If det(A*^{G}*) is identically zero, then it*

*remains zero if we substitute arbitrary integers for the*
*variables x*_{11}*, . . . , x** _{nn}*.

*•* *What is the likelihood of obtaining a zero when det(A** ^{G}*)

*is not identically zero?*

### Number of Roots of a Polynomial

Lemma 59 (Schwartz (1980)) *Let p(x*_{1}*, x*_{2}*, . . . , x*_{m}*) 6≡ 0*
*be a polynomial in m variables each of degree at most d. Let*
*M ∈ Z*^{+}*. Then the number of m-tuples*

*(x*_{1}*, x*_{2}*, . . . , x*_{m}*) ∈ {0, 1, . . . , M − 1}*^{m}*such that p(x*_{1}*, x*_{2}*, . . . , x*_{m}*) = 0 is*

*≤ mdM*^{m−1}*.*

*•* *By induction on m (consult the textbook).*

### Density Attack

*•* The density of roots in the domain is at most
*mdM*^{m−1}

*M** ^{m}* =

*md*

*M* *.* (6)

*•* *So suppose p(x*_{1}*, x*_{2}*, . . . , x*_{m}*) 6≡ 0.*

*•* Then a random

*(x*_{1}*, x*_{2}*, . . . , x*_{m}*) ∈ { 0, 1, . . . , M − 1 }*^{m}*has a probability of ≤ md/M of being a root of p.*

*•* *Note that M is under our control.*

### Density Attack (concluded)

*Here is a sampling algorithm to test if p(x*_{1}*, x*_{2}*, . . . , x*_{m}*) 6≡ 0.*

1: *Choose i*_{1}*, . . . , i*_{m}*from {0, 1, . . . , M − 1} randomly;*

2: *if p(i*_{1}*, i*_{2}*, . . . , i*_{m}*) 6= 0 then*

3: *return “p is not identically zero”;*

4: else

5: *return “p is identically zero”;*

6: end if

### A Randomized Bipartite Perfect Matching Algorithm

^{a}

We now return to the original problem of bipartite perfect matching.

1: *Choose n*^{2} *integers i*_{11}*, . . . , i*_{nn}*from {0, 1, . . . , 2n*^{2} *− 1}*

randomly;

2: *Calculate det(A*^{G}*(i*_{11}*, . . . , i** _{nn}*)) by Gaussian elimination;

3: *if det(A*^{G}*(i*_{11}*, . . . , i*_{nn}*)) 6= 0 then*

4: *return “G has a perfect matching”;*

5: else

6: *return “G has no perfect matchings”;*

7: end if

aLov´asz (1979). According to Paul Erd˝os, Lov´asz wrote his first sig- nificant paper “at the ripe old age of 17.”

### Analysis

*•* *If G has no perfect matchings, the algorithm will always*
be correct.

*•* *Suppose G has a perfect matching.*

– The algorithm will answer incorrectly with

*probability at most n*^{2}*d/(2n*^{2}*) = 0.5 with d = 1 in*
Eq. (6) on p. 431.

– *Run the algorithm independently k times and output*

*“G has no perfect matchings” if they all say no.*

– The error probability is now reduced to at most 2* ^{−k}*.

*•* *Is there an (i*_{11}*, . . . , i** _{nn}*) that will always give correct

*answers for all bipartite graphs of 2n nodes?*

^{a}

aThanks to a lively class discussion on November 24, 2004.

### Analysis (concluded)

^{a}

*•* Note that we are calculating

*prob[ algorithm answers “yes” | G has a prefect matching ],*
*prob[ algorithm answers “no” | G has no prefect matchings ].*

*•* *We are not calculating*

*prob[ G has a prefect matching | algorithm answers “yes” ],*
*prob[ G has no prefect matchings | algorithm answers “no” ].*

aThanks to a lively class discussion on May 1, 2008.

### L´oszl´o Lov´asz (1948–)

### Perfect Matching for General Graphs

*•* Page 423 is about bipartite perfect matching

*•* *Now we are given a graph G = (V, E).*

– *V = {v*_{1}*, v*_{2}*, . . . , v*_{2n}*}.*

*•* We are asked if there is a perfect matching.

– *A permutation π of {1, 2, . . . , 2n} such that*
*(v*_{i}*, v*_{π(i)}*) ∈ E*

*for all v*_{i}*∈ V .*

### The Tutte Matrix

^{a}

*•* *Given a graph G = (V, E), construct the 2n × 2n Tutte*
*matrix T** ^{G}* such that

*T*_{ij}* ^{G}* =

*x*_{ij}*if (v*_{i}*, v*_{j}*) ∈ E and i < j,*

*−x*_{ij}*if (v*_{i}*, v*_{j}*) ∈ E and i > j,*
0 *othersie.*

*•* The Tutte matrix is a skew-symmetric symbolic matrix.

*•* Similar to Proposition 58 (p. 426):

Proposition 60 *G has a perfect matching if and only if*
*det(T*^{G}*) is not identically zero.*

aWilliam Thomas Tutte (1917–2002).

### William Thomas Tutte (1917–2002)

### Monte Carlo Algorithms

^{a}

*•* The randomized bipartite perfect matching algorithm is
called a Monte Carlo algorithm in the sense that

– If the algorithm finds that a matching exists, it is always correct (no false positives).

– If the algorithm answers in the negative, then it may make an error (false negative).

aMetropolis and Ulam (1949).

### Monte Carlo Algorithms (concluded)

*•* The algorithm makes a false negative with probability

*≤ 0.5.*

– Note this probability refers to

*prob[ algorithm answers “no” | G has a prefect matching ].*

*•* *This probability is not over the space of all graphs or*
*determinants, but over the algorithm’s own coin flips.*

– *It holds for any bipartite graph.*

### False Positives and False Negatives in Human Behavior?

^{a}

*•* “[Men] tend to misinterpret innocent friendliness as a
*sign that women are [· · · ] interested in them.”*

– A false positive.

*•* “[Women] tend to undervalue signs that a man is
interested in a committed relationship.”

– A false negative.

a*“Don’t misunderestimate yourself.” The Economist, 2006.*

### The Markov Inequality

^{a}

Lemma 61 *Let x be a random variable taking nonnegative*
*integer values. Then for any k > 0,*

*prob[x ≥ kE[ x ]] ≤ 1/k.*

*•* *Let p*_{i}*denote the probability that x = i.*

*E[ x ] =* X

*i*

*ip*_{i}

= X

*i<kE[ x ]*

*ip** _{i}* + X

*i≥kE[ x ]*

*ip*_{i}

*≥ kE[ x ] × prob[x ≥ kE[ x ]].*

aAndrei Andreyevich Markov (1856–1922).

### Andrei Andreyevich Markov (1856–1922)

### An Application of Markov’s Inequality

*•* *Algorithm C runs in expected time T (n) and always*
gives the right answer.

*•* *Consider an algorithm that runs C for time kT (n) and*
*rejects the input if C does not stop within the time*
bound.

*•* By Markov’s inequality, this new algorithm runs in time
*kT (n) and gives the wrong answer with probability*

*≤ 1/k.*

*•* *By running this algorithm m times, we reduce the error*
*probability to ≤ k** ^{−m}*.

### An Application of Markov’s Inequality (concluded)

*•* Suppose, instead, we run the algorithm for the same
*running time mkT (n) once and rejects the input if it*
does not stop within the time bound.

*•* By Markov’s inequality, this new algorithm gives the
*wrong answer with probability ≤ 1/(mk).*

*•* This is a far cry from the previous algorithm’s error
*probability of ≤ k** ^{−m}*.

*•* The loss comes from the fact that Markov’s inequality
does not take advantage of any specific feature of the
random variable.

*fsat for k-sat Formulas (p. 411)*

*•* *Let φ(x*_{1}*, x*_{2}*, . . . , x*_{n}*) be a k-sat formula.*

*•* *If φ is satisfiable, then return a satisfying truth*
assignment.

*•* Otherwise, return “no.”

*•* We next propose a randomized algorithm for this
problem.

*A Random Walk Algorithm for φ in CNF Form*

1: *Start with an arbitrary truth assignment T ;*

2: *for i = 1, 2, . . . , r do*

3: *if T |= φ then*

4: *return “φ is satisfiable with T ”;*

5: else

6: *Let c be an unsatisfiable clause in φ under T ; {All*
*of its literals are false under T .}*

7: *Pick any x of these literals at random;*

8: *Modify T to make x true;*

9: end if

10: end for

11: *return “φ is unsatisfiable”;*

### 3sat vs. 2sat Again

*•* *Note that if φ is unsatisfiable, the algorithm will not*
refute it.

*•* The random walk algorithm needs expected exponential
time for 3sat.

– *In fact, it runs in expected O((1.333 · · · + ²)** ^{n}*) time

*with r = 3n,*

^{a}

*much better than O(2*

*).*

^{n}^{b}

*•* We will show immediately that it works well for 2sat.

*•* *The state of the art as of 2006 is expected O(1.322** ^{n}*)

*time for 3sat and expected O(1.474*

*) time for 4sat.*

^{n}^{c}

aUse this setting per run of the algorithm.

bSch¨oning (1999).

cKwama and Tamaki (2004); Rolf (2006).

### Random Walk Works for 2sat

^{a}

Theorem 62 *Suppose the random walk algorithm with*
*r = 2n*^{2} *is applied to any satisfiable 2sat problem with n*
*variables. Then a satisfying truth assignment will be*

*discovered with probability at least 0.5.*

*•* Let ˆ*T be a truth assignment such that ˆT |= φ.*

*•* *Let t(i) denote the expected number of repetitions of the*
flipping step until a satisfying truth assignment is found
*if our starting T differs from ˆT in i values.*

– *Their Hamming distance is i.*

aPapadimitriou (1991).

### The Proof

*•* *It can be shown that t(i) is finite.*

*•* *t(0) = 0 because it means that T = ˆT and hence T |= φ.*

*•* *If T 6= ˆT or T is not equal to any other satisfying truth*
assignment, then we need to flip at least once.

*•* We flip to pick among the 2 literals of a clause not
*satisfied by the present T .*

*•* At least one of the 2 literals is true under ˆ*T because ˆT*
satisfies all clauses.

*•* So we have at least 0.5 chance of moving closer to ˆ*T .*

### The Proof (continued)

*•* Thus

*t(i) ≤* *t(i − 1) + t(i + 1)*

2 + 1

*for 0 < i < n.*

– *Inequality is used because, for example, T may differ*
from ˆ*T in both literals.*

*•* It must also hold that

*t(n) ≤ t(n − 1) + 1*
*because at i = n, we can only decrease i.*

### The Proof (continued)

*•* As we are only interested in upper bounds, we solve
*x(0) = 0*

*x(n) = x(n − 1) + 1*

*x(i) =* *x(i − 1) + x(i + 1)*

2 *+ 1, 0 < i < n*

*•* This is one-dimensional random walk with a reflecting
and an absorbing barrier.

### The Proof (continued)

*•* Add the equations up to obtain

*x(1) + x(2) + · · · + x(n)*

= *x(0) + x(1) + 2x(2) + · · · + 2x(n − 2) + x(n − 1) + x(n)*
2

*+n + x(n − 1).*

*•* Simplify to yield

*x(1) + x(n) − x(n − 1)*

2 *= n.*

*•* *As x(n) − x(n − 1) = 1, we have*
*x(1) = 2n − 1.*

### The Proof (continued)

*•* Iteratively, we obtain

*x(2) = 4n − 4,*
...

*x(i) = 2in − i*^{2}*.*

*•* *The worst case happens when i = n, in which case*
*x(n) = n*^{2}*.*

### The Proof (concluded)

*•* We therefore reach the conclusion that
*t(i) ≤ x(i) ≤ x(n) = n*^{2}*.*

*•* *So the expected number of steps is at most n*^{2}.

*•* *The algorithm picks a running time 2n*^{2}.

*•* This amounts to invoking the Markov inequality (p. 443)
*with k = 2, with the consequence of having a probability*
of 0.5.

*•* The proof does not yield a polynomial bound for 3sat.^{a}

aContributed by Mr. Cheng-Yu Lee (R95922035) on November 8, 2006.

### Boosting the Performance

*•* *We can pick r = 2mn*^{2} to have an error probability of

*≤ (2m)** ^{−1}* by Markov’s inequality.

*•* Alternatively, with the same running time, we can run
*the “r = 2n*^{2}*” algorithm m times.*

*•* *But the error probability is reduced to ≤ 2** ^{−m}*!

*•* Again, the gain comes from the fact that Markov’s
inequality does not take advantage of any specific
feature of the random variable.

*•* The gain also comes from the fact that the two
algorithms are different.

### Primality Tests

*•* *primes asks if a number N is a prime.*

*•* *The classic algorithm tests if k | N for k = 2, 3, . . . ,√*
*N .*

*•* But it runs in Ω(2^{n/2}*) steps, where n = | N | = log*_{2} *N .*

### The Density Attack for primes

1: *Pick k ∈ {2, . . . , N − 1} randomly; {Assume N > 2.}*

2: *if k | N then*

3: *return “N is composite”;*

4: else

5: *return “N is a prime”;*

6: end if

### Analysis

^{a}

*•* *Suppose N = P Q, a product of 2 primes.*

*•* The probability of success is

*< 1 −* *φ(N )*

*N* *= 1 −* *(P − 1)(Q − 1)*

*P Q* = *P + Q − 1*

*P Q* *.*

*•* *In the case where P ≈ Q, this probability becomes*

*<* 1

*P* + 1

*Q* *≈* 2

*√N* *.*

*•* This probability is exponentially small.

aSee also p. 394.

### The Fermat Test for Primality

Fermat’s “little” theorem on p. 396 suggests the following
*primality test for any given number p:*

1: *Pick a number a randomly from {1, 2, . . . , N − 1};*

2: *if a*^{N −1}*6= 1 mod N then*

3: *return “N is composite”;*

4: else

5: *return “N is a prime”;*

6: end if

### The Fermat Test for Primality (concluded)

*•* Unfortunately, there are composite numbers called
Carmichael numbers that will pass the Fermat test
*for all a ∈ {1, 2, . . . , N − 1}.*^{a}

*•* There are infinitely many Carmichael numbers.^{b}

*•* *In fact, the number of Carmichael numbers less than n*
*exceeds n*^{2/7}*for n large enough.*

aCarmichael (1910).

bAlford, Granville, and Pomerance (1992).

### Square Roots Modulo a Prime

*•* *Equation x*^{2} *= a mod p has at most two (distinct) roots*
by Lemma 56 (p. 401).

– The roots are called square roots.

– *Numbers a with square roots and gcd(a, p) = 1 are*
called quadratic residues.

*∗* They are 1^{2} *mod p, 2*^{2} *mod p, . . . , (p − 1)*^{2} *mod p.*

*•* We shall show that a number either has two roots or has
none, and testing which one is true is trivial.

*•* *There are no known efficient deterministic algorithms to*
find the roots, however.

### Euler’s Test

Lemma 63 (Euler) *Let p be an odd prime and*
*a 6= 0 mod p.*

*1.* *If a*^{(p−1)/2}*= 1 mod p, then x*^{2} *= a mod p has two roots.*

*2.* *If a*^{(p−1)/2}*6= 1 mod p, then a*^{(p−1)/2}*= −1 mod p and*
*x*^{2} *= a mod p has no roots.*

*•* *Let r be a primitive root of p.*

*•* *By Fermat’s “little” theorem, r** ^{(p−1)/2}* is a square root of

*1, so r*

^{(p−1)/2}*= 1 mod p or r*

^{(p−1)/2}*= −1 mod p.*

*•* *But as r is a primitive root, r*^{(p−1)/2}*6= 1 mod p.*

*•* *Hence r*^{(p−1)/2}*= −1 mod p.*

### The Proof (continued)

*•* *Let a = r*^{k}*mod p for some k.*

*•* Then

*a*^{(p−1)/2}*= r** ^{k(p−1)/2}* = £

*r** ^{(p−1)/2}* ¤

_{k}*= (−1)*^{k}*= 1 mod p.*

*•* *So k must be even.*

*•* *Suppose a = r*^{2j}*for some 1 ≤ j ≤ (p − 1)/2.*

*•* *Then a*^{(p−1)/2}*= r*^{j(p−1)}*= 1 mod p and its two distinct*
*roots are r*^{j}*, −r*^{j}*(= r** ^{j+(p−1)/2}*).

– *If r*^{j}*= −r*^{j}*mod p, then 2r*^{j}*= 0 mod p, which implies*
*r*^{j}*= 0 mod p, a contradiction.*

### The Proof (continued)

*•* *As 1 ≤ j ≤ (p − 1)/2, there are (p − 1)/2 such a’s.*

*•* *Each such a has 2 distinct square roots.*

*•* *The square roots of all the a’s are distinct.*

– *The square roots of different a’s must be different.*

*•* *Hence the set of square roots is {1, 2, . . . , p − 1}.*

– *Because there are (p − 1)/2 such a’s and each a has*
two square roots.

*•* *As a result, a = r*^{2j}*, 1 ≤ j ≤ (p − 1)/2, are all the*
quadratic residues.

### The Proof (concluded)

*•* *If a = r** ^{2j+1}*, then it has no roots because all the square
roots have been taken.

*•* Now,

*a** ^{(p−1)/2}* =
h

*r*^{(p−1)/2}

i_{2j+1}

*= (−1)*^{2j+1}*= −1 mod p.*

The Legendre Symbol^{a} and Quadratic Residuacity Test

*•* *By Lemma 63 (p. 464) a*^{(p−1)/2}*mod p = ±1 for*
*a 6= 0 mod p.*

*•* *For odd prime p, define the Legendre symbol (a | p) as*

*(a | p) =*

0 *if p | a,*

1 *if a is a quadratic residue modulo p,*

*−1* *if a is a quadratic nonresidue modulo p.*

*•* *Euler’s test implies a*^{(p−1)/2}*= (a | p) mod p for any odd*
*prime p and any integer a.*

*•* *Note that (ab|p) = (a|p)(b|p).*

aAndrien-Marie Legendre (1752–1833).

### Gauss’s Lemma

Lemma 64 (Gauss) *Let p and q be two odd primes. Then*
*(q|p) = (−1)*^{m}*, where m is the number of residues in*

*R = {iq mod p : 1 ≤ i ≤ (p − 1)/2} that are greater than*
*(p − 1)/2.*

*•* *All residues in R are distinct.*

– *If iq = jq mod p, then p|(j − i) q or p|q.*

*•* *No two elements of R add up to p.*

– *If iq + jq = 0 mod p, then p|(i + j) or p|q.*

– But neither is possible.

### The Proof (continued)

*•* *Consider the set R*^{0}*of residues that result from R if we*
*replace each of the m elements a ∈ R such that*

*a > (p − 1)/2 by p − a.*

– *This is equivalent to performing −a mod p.*

*•* *All residues in R*^{0}*are now at most (p − 1)/2.*

*•* *In fact, R*^{0}*= {1, 2, . . . , (p − 1)/2} (see illustration next*
page).

– *Otherwise, two elements of R would add up to p,*
which has been shown to be impossible.

5 1 2 3 4

6 5

1 2 3 4

6

*p = 7 and q = 5.*

### The Proof (concluded)

*•* *Alternatively, R*^{0}*= {±iq mod p : 1 ≤ i ≤ (p − 1)/2},*
*where exactly m of the elements have the minus sign.*

*•* Take the product of all elements in the two
*representations of R** ^{0}*.

*•* *So [(p − 1)/2]! = (−1)*^{m}*q*^{(p−1)/2}*[(p − 1)/2]! mod p.*

*•* *Because gcd([(p − 1)/2]!, p) = 1, the above implies*
*1 = (−1)*^{m}*q*^{(p−1)/2}*mod p.*