A Perfect Matching

Randomized Computation

I know that half my advertising works, I just don’t know which half.

— John Wanamaker

I know that half my advertising is a waste of money, I just don’t know which half!

— McGraw-Hill ad.

Randomized Algorithms

• Randomized algorithms flip unbiased coins.

• There are important problems for which there are no known efficient deterministic algorithms but for which very efficient randomized algorithms exist.

– Extraction of square roots, for instance.

• There are problems where randomization is necessary.

– Secure protocols.

• Randomized version can be more efficient.

– Parallel algorithm for maximal independent set.

aRabin (1976); Solovay and Strassen (1977).

“Four Most Important Randomized Algorithms”

1. Primality testing.^b

2. Graph connectivity using random walks.^c 3. Polynomial identity testing.^d

4. Algorithms for approximate counting.^e

aTrevisan (2006).

bRabin (1976); Solovay and Strassen (1977).

cAleliunas, Karp, Lipton, Lov´asz, and Rackoff (1979).

dSchwartz (1980); Zippel (1979).

eSinclair and Jerrum (1989).

Bipartite Perfect Matching

• We are given a bipartite graph G = (U, V, E).

– U = {u₁, u₂, . . . , u_n}.

– V = {v₁, v₂, . . . , v_n}.

– E ⊆ U × V .

• We are asked if there is a perfect matching.

– A permutation π of {1, 2, . . . , n} such that (u_i, v_π(i)) ∈ E

for all u_i ∈ U .

A Perfect Matching

X_

X_

X_

X_

X_

Y_

Y_

Y_

Y_

Y_

Symbolic Determinants

• Given a bipartite graph G, construct the n × n matrix A^G whose (i, j)th entry A^G_ij is a variable x_ij if

(u_i, v_j) ∈ E and zero otherwise.

• The determinant of A^G is det(A^G) = X

π

sgn(π) Yn i=1

A^G_i,π(i). (5) – π ranges over all permutations of n elements.

– sgn(π) is 1 if π is the product of an even number of transpositions and −1 otherwise.

– Equivalently, sgn(π) = 1 if the number of (i, j)s such that i < j) and π(i) > π(j) is even.^a

Determinant and Bipartite Perfect Matching

• In P

π sgn(π)Q_n

i=1 A^G_i,π(i), note the following:

– Each summand corresponds to a possible prefect matching π.

– As all variables appear only once, all of these summands are different monomials and will not cancel.

• It is essentially an exhaustive enumeration.

Proposition 58 (Edmonds (1967)) G has a perfect matching if and only if det(A^G) is not identically zero.

A Perfect Matching in a Bipartite Graph

X_

X_

X_

X_

X_

Y_

Y_

Y_

Y_

Y_

The Perfect Matching in the Determinant

• The matrix is

A^G =













0 0 x₁₃ x₁₄ 0

0 x₂₂ 0 0 0

x₃₁ 0 0 0 x₃₅

x₄₁ 0 x₄₃ x₄₄ 0

x₅₁ 0 0 0 x₅₅











 .

• det(A^G) = −x₁₄x₂₂x₃₅x₄₃x₅₁ + x₁₃x₂₂x₃₅x₄₄x₅₁ + x₁₄x₂₂x₃₁x₄₃x₅₅ − x₁₃x₂₂x₃₁x₄₄x₅₅, each denoting a perfect matching.

How To Test If a Polynomial Is Identically Zero?

• det(A^G) is a polynomial in n² variables.

• There are exponentially many terms in det(A^G).

• Expanding the determinant polynomial is not feasible.

– Too many terms.

• Observation: If det(A^G) is identically zero, then it

remains zero if we substitute arbitrary integers for the variables x₁₁, . . . , x_nn.

• What is the likelihood of obtaining a zero when det(A^G) is not identically zero?

Number of Roots of a Polynomial

Lemma 59 (Schwartz (1980)) Let p(x₁, x₂, . . . , x_m) 6≡ 0 be a polynomial in m variables each of degree at most d. Let M ∈ Z⁺. Then the number of m-tuples

(x₁, x₂, . . . , x_m) ∈ {0, 1, . . . , M − 1}^m such that p(x₁, x₂, . . . , x_m) = 0 is

≤ mdM^m−1.

• By induction on m (consult the textbook).

Density Attack

• The density of roots in the domain is at most mdM^m−1

M^m = md

M . (6)

• So suppose p(x₁, x₂, . . . , x_m) 6≡ 0.

• Then a random

(x₁, x₂, . . . , x_m) ∈ { 0, 1, . . . , M − 1 }^m has a probability of ≤ md/M of being a root of p.

• Note that M is under our control.

Density Attack (concluded)

Here is a sampling algorithm to test if p(x₁, x₂, . . . , x_m) 6≡ 0.

1: Choose i₁, . . . , i_m from {0, 1, . . . , M − 1} randomly;

2: if p(i₁, i₂, . . . , i_m) 6= 0 then

3: return “p is not identically zero”;

4: else

5: return “p is identically zero”;

6: end if

A Randomized Bipartite Perfect Matching Algorithm

We now return to the original problem of bipartite perfect matching.

1: Choose n² integers i₁₁, . . . , i_nn from {0, 1, . . . , 2n² − 1}

randomly;

2: Calculate det(A^G(i₁₁, . . . , i_nn)) by Gaussian elimination;

3: if det(A^G(i₁₁, . . . , i_nn)) 6= 0 then

4: return “G has a perfect matching”;

5: else

6: return “G has no perfect matchings”;

7: end if

aLov´asz (1979). According to Paul Erd˝os, Lov´asz wrote his first sig- nificant paper “at the ripe old age of 17.”

Analysis

• If G has no perfect matchings, the algorithm will always be correct.

• Suppose G has a perfect matching.

– The algorithm will answer incorrectly with

probability at most n²d/(2n²) = 0.5 with d = 1 in Eq. (6) on p. 431.

– Run the algorithm independently k times and output

“G has no perfect matchings” if they all say no.

– The error probability is now reduced to at most 2^−k.

• Is there an (i₁₁, . . . , i_nn) that will always give correct answers for all bipartite graphs of 2n nodes?^a

aThanks to a lively class discussion on November 24, 2004.

Analysis (concluded)

• Note that we are calculating

prob[ algorithm answers “yes” | G has a prefect matching ], prob[ algorithm answers “no” | G has no prefect matchings ].

• We are not calculating

prob[ G has a prefect matching | algorithm answers “yes” ], prob[ G has no prefect matchings | algorithm answers “no” ].

aThanks to a lively class discussion on May 1, 2008.

L´oszl´o Lov´asz (1948–)

Perfect Matching for General Graphs

• Page 423 is about bipartite perfect matching

• Now we are given a graph G = (V, E).

– V = {v₁, v₂, . . . , v_2n}.

• We are asked if there is a perfect matching.

– A permutation π of {1, 2, . . . , 2n} such that (v_i, v_π(i)) ∈ E

for all v_i ∈ V .

The Tutte Matrix

• Given a graph G = (V, E), construct the 2n × 2n Tutte matrix T^G such that

T_ij^G =









x_ij if (v_i, v_j) ∈ E and i < j,

−x_ij if (v_i, v_j) ∈ E and i > j, 0 othersie.

• The Tutte matrix is a skew-symmetric symbolic matrix.

• Similar to Proposition 58 (p. 426):

Proposition 60 G has a perfect matching if and only if det(T^G) is not identically zero.

aWilliam Thomas Tutte (1917–2002).

William Thomas Tutte (1917–2002)

Monte Carlo Algorithms

• The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that

– If the algorithm finds that a matching exists, it is always correct (no false positives).

– If the algorithm answers in the negative, then it may make an error (false negative).

aMetropolis and Ulam (1949).

Monte Carlo Algorithms (concluded)

• The algorithm makes a false negative with probability

≤ 0.5.

– Note this probability refers to

prob[ algorithm answers “no” | G has a prefect matching ].

• This probability is not over the space of all graphs or determinants, but over the algorithm’s own coin flips.

– It holds for any bipartite graph.

False Positives and False Negatives in Human Behavior?

• “[Men] tend to misinterpret innocent friendliness as a sign that women are [· · · ] interested in them.”

– A false positive.

• “[Women] tend to undervalue signs that a man is interested in a committed relationship.”

– A false negative.

a“Don’t misunderestimate yourself.” The Economist, 2006.

The Markov Inequality

Lemma 61 Let x be a random variable taking nonnegative integer values. Then for any k > 0,

prob[x ≥ kE[ x ]] ≤ 1/k.

• Let p_i denote the probability that x = i.

E[ x ] = X

i

ip_i

= X

i<kE[ x ]

ip_i + X

i≥kE[ x ]

ip_i

≥ kE[ x ] × prob[x ≥ kE[ x ]].

aAndrei Andreyevich Markov (1856–1922).

Andrei Andreyevich Markov (1856–1922)

An Application of Markov’s Inequality

• Algorithm C runs in expected time T (n) and always gives the right answer.

• Consider an algorithm that runs C for time kT (n) and rejects the input if C does not stop within the time bound.

• By Markov’s inequality, this new algorithm runs in time kT (n) and gives the wrong answer with probability

≤ 1/k.

• By running this algorithm m times, we reduce the error probability to ≤ k^−m.

An Application of Markov’s Inequality (concluded)

• Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound.

• By Markov’s inequality, this new algorithm gives the wrong answer with probability ≤ 1/(mk).

• This is a far cry from the previous algorithm’s error probability of ≤ k^−m.

• The loss comes from the fact that Markov’s inequality does not take advantage of any specific feature of the random variable.

fsat for k-sat Formulas (p. 411)

• Let φ(x₁, x₂, . . . , x_n) be a k-sat formula.

• If φ is satisfiable, then return a satisfying truth assignment.

• Otherwise, return “no.”

• We next propose a randomized algorithm for this problem.

A Random Walk Algorithm for φ in CNF Form

1: Start with an arbitrary truth assignment T ;

2: for i = 1, 2, . . . , r do

3: if T |= φ then

4: return “φ is satisfiable with T ”;

5: else

6: Let c be an unsatisfiable clause in φ under T ; {All of its literals are false under T .}

7: Pick any x of these literals at random;

8: Modify T to make x true;

9: end if

10: end for

11: return “φ is unsatisfiable”;

3sat vs. 2sat Again

• Note that if φ is unsatisfiable, the algorithm will not refute it.

• The random walk algorithm needs expected exponential time for 3sat.

– In fact, it runs in expected O((1.333 · · · + ²)ⁿ) time with r = 3n,^a much better than O(2ⁿ).^b

• We will show immediately that it works well for 2sat.

• The state of the art as of 2006 is expected O(1.322ⁿ) time for 3sat and expected O(1.474ⁿ) time for 4sat.^c

aUse this setting per run of the algorithm.

bSch¨oning (1999).

cKwama and Tamaki (2004); Rolf (2006).

Random Walk Works for 2sat

Theorem 62 Suppose the random walk algorithm with r = 2n² is applied to any satisfiable 2sat problem with n variables. Then a satisfying truth assignment will be

discovered with probability at least 0.5.

• Let ˆT be a truth assignment such that ˆT |= φ.

• Let t(i) denote the expected number of repetitions of the flipping step until a satisfying truth assignment is found if our starting T differs from ˆT in i values.

– Their Hamming distance is i.

aPapadimitriou (1991).

The Proof

• It can be shown that t(i) is finite.

• t(0) = 0 because it means that T = ˆT and hence T |= φ.

• If T 6= ˆT or T is not equal to any other satisfying truth assignment, then we need to flip at least once.

• We flip to pick among the 2 literals of a clause not satisfied by the present T .

• At least one of the 2 literals is true under ˆT because ˆT satisfies all clauses.

• So we have at least 0.5 chance of moving closer to ˆT .

The Proof (continued)

• Thus

t(i) ≤ t(i − 1) + t(i + 1)

2 + 1

for 0 < i < n.

– Inequality is used because, for example, T may differ from ˆT in both literals.

• It must also hold that

t(n) ≤ t(n − 1) + 1 because at i = n, we can only decrease i.

The Proof (continued)

• As we are only interested in upper bounds, we solve x(0) = 0

x(n) = x(n − 1) + 1

x(i) = x(i − 1) + x(i + 1)

2 + 1, 0 < i < n

• This is one-dimensional random walk with a reflecting and an absorbing barrier.

The Proof (continued)

• Add the equations up to obtain

x(1) + x(2) + · · · + x(n)

= x(0) + x(1) + 2x(2) + · · · + 2x(n − 2) + x(n − 1) + x(n) 2

+n + x(n − 1).

• Simplify to yield

x(1) + x(n) − x(n − 1)

2 = n.

• As x(n) − x(n − 1) = 1, we have x(1) = 2n − 1.

The Proof (continued)

• Iteratively, we obtain

x(2) = 4n − 4, ...

x(i) = 2in − i².

• The worst case happens when i = n, in which case x(n) = n².

The Proof (concluded)

• We therefore reach the conclusion that t(i) ≤ x(i) ≤ x(n) = n².

• So the expected number of steps is at most n².

• The algorithm picks a running time 2n².

• This amounts to invoking the Markov inequality (p. 443) with k = 2, with the consequence of having a probability of 0.5.

• The proof does not yield a polynomial bound for 3sat.^a

aContributed by Mr. Cheng-Yu Lee (R95922035) on November 8, 2006.

Boosting the Performance

• We can pick r = 2mn² to have an error probability of

≤ (2m)⁻¹ by Markov’s inequality.

• Alternatively, with the same running time, we can run the “r = 2n²” algorithm m times.

• But the error probability is reduced to ≤ 2^−m!

• Again, the gain comes from the fact that Markov’s inequality does not take advantage of any specific feature of the random variable.

• The gain also comes from the fact that the two algorithms are different.

Primality Tests

• primes asks if a number N is a prime.

• The classic algorithm tests if k | N for k = 2, 3, . . . ,√ N .

• But it runs in Ω(2^n/2) steps, where n = | N | = log₂ N .

The Density Attack for primes

1: Pick k ∈ {2, . . . , N − 1} randomly; {Assume N > 2.}

2: if k | N then

3: return “N is composite”;

4: else

5: return “N is a prime”;

6: end if

Analysis

• Suppose N = P Q, a product of 2 primes.

• The probability of success is

< 1 − φ(N )

N = 1 − (P − 1)(Q − 1)

P Q = P + Q − 1

P Q .

• In the case where P ≈ Q, this probability becomes

< 1

P + 1

Q ≈ 2

√N .

• This probability is exponentially small.

aSee also p. 394.

The Fermat Test for Primality

Fermat’s “little” theorem on p. 396 suggests the following primality test for any given number p:

1: Pick a number a randomly from {1, 2, . . . , N − 1};

2: if a^{N −1} 6= 1 mod N then

3: return “N is composite”;

4: else

5: return “N is a prime”;

6: end if

The Fermat Test for Primality (concluded)

• Unfortunately, there are composite numbers called Carmichael numbers that will pass the Fermat test for all a ∈ {1, 2, . . . , N − 1}.^a

• There are infinitely many Carmichael numbers.^b

• In fact, the number of Carmichael numbers less than n exceeds n^2/7 for n large enough.

aCarmichael (1910).

bAlford, Granville, and Pomerance (1992).

Square Roots Modulo a Prime

• Equation x² = a mod p has at most two (distinct) roots by Lemma 56 (p. 401).

– The roots are called square roots.

– Numbers a with square roots and gcd(a, p) = 1 are called quadratic residues.

∗ They are 1² mod p, 2² mod p, . . . , (p − 1)² mod p.

• We shall show that a number either has two roots or has none, and testing which one is true is trivial.

• There are no known efficient deterministic algorithms to find the roots, however.

Euler’s Test

Lemma 63 (Euler) Let p be an odd prime and a 6= 0 mod p.

1. If a^(p−1)/2 = 1 mod p, then x² = a mod p has two roots.

2. If a^(p−1)/2 6= 1 mod p, then a^(p−1)/2 = −1 mod p and x² = a mod p has no roots.

• Let r be a primitive root of p.

• By Fermat’s “little” theorem, r^(p−1)/2 is a square root of 1, so r^(p−1)/2 = 1 mod p or r^(p−1)/2 = −1 mod p.

• But as r is a primitive root, r^(p−1)/2 6= 1 mod p.

• Hence r^(p−1)/2 = −1 mod p.

The Proof (continued)

• Let a = r^k mod p for some k.

• Then

a^(p−1)/2 = r^k(p−1)/2 = £

r^(p−1)/2 ¤_k

= (−1)^k = 1 mod p.

• So k must be even.

• Suppose a = r^2j for some 1 ≤ j ≤ (p − 1)/2.

• Then a^(p−1)/2 = r^j(p−1) = 1 mod p and its two distinct roots are r^j, −r^j(= r^j+(p−1)/2).

– If r^j = −r^j mod p, then 2r^j = 0 mod p, which implies r^j = 0 mod p, a contradiction.

The Proof (continued)

• As 1 ≤ j ≤ (p − 1)/2, there are (p − 1)/2 such a’s.

• Each such a has 2 distinct square roots.

• The square roots of all the a’s are distinct.

– The square roots of different a’s must be different.

• Hence the set of square roots is {1, 2, . . . , p − 1}.

– Because there are (p − 1)/2 such a’s and each a has two square roots.

• As a result, a = r^2j, 1 ≤ j ≤ (p − 1)/2, are all the quadratic residues.

The Proof (concluded)

• If a = r^2j+1, then it has no roots because all the square roots have been taken.

• Now,

a^(p−1)/2 = h

r^(p−1)/2

i_2j+1

= (−1)^2j+1 = −1 mod p.

The Legendre Symbol^a and Quadratic Residuacity Test

• By Lemma 63 (p. 464) a^(p−1)/2 mod p = ±1 for a 6= 0 mod p.

• For odd prime p, define the Legendre symbol (a | p) as

(a | p) =









0 if p | a,

1 if a is a quadratic residue modulo p,

−1 if a is a quadratic nonresidue modulo p.

• Euler’s test implies a^(p−1)/2 = (a | p) mod p for any odd prime p and any integer a.

• Note that (ab|p) = (a|p)(b|p).

aAndrien-Marie Legendre (1752–1833).

Gauss’s Lemma

Lemma 64 (Gauss) Let p and q be two odd primes. Then (q|p) = (−1)^m, where m is the number of residues in

R = {iq mod p : 1 ≤ i ≤ (p − 1)/2} that are greater than (p − 1)/2.

• All residues in R are distinct.

– If iq = jq mod p, then p|(j − i) q or p|q.

• No two elements of R add up to p.

– If iq + jq = 0 mod p, then p|(i + j) or p|q.

– But neither is possible.

The Proof (continued)

• Consider the set R⁰ of residues that result from R if we replace each of the m elements a ∈ R such that

a > (p − 1)/2 by p − a.

– This is equivalent to performing −a mod p.

• All residues in R⁰ are now at most (p − 1)/2.

• In fact, R⁰ = {1, 2, . . . , (p − 1)/2} (see illustration next page).

– Otherwise, two elements of R would add up to p, which has been shown to be impossible.

5 1 2 3 4

6 5

1 2 3 4

6

p = 7 and q = 5.

The Proof (concluded)

• Alternatively, R⁰ = {±iq mod p : 1 ≤ i ≤ (p − 1)/2}, where exactly m of the elements have the minus sign.

• Take the product of all elements in the two representations of R⁰.

• So [(p − 1)/2]! = (−1)^mq^(p−1)/2[(p − 1)/2]! mod p.

• Because gcd([(p − 1)/2]!, p) = 1, the above implies 1 = (−1)^mq^(p−1)/2 mod p.