Bipartite Perfect Matching

Bipartite Perfect Matching

• We are given a bipartite graph G = (U, V, E).

– U = {u¹, u₂, . . . , u_n}.

– V = {v¹, v₂, . . . , v_n}.

– E ⊆ U × V .

• We are asked if there is a perfect matching.

– A permutation π of {1, 2, . . . , n} such that (u_i, v_π(i)) ∈ E

for all i ∈ {1, 2, . . . , n}.

A Perfect Matching in a Bipartite Graph

X_

X_

X_

X_

Y_

Y_

Y_

Y_

Symbolic Determinants

• We are given a bipartite graph G.

• Construct the n × n matrix A^G whose (i, j)th entry A^G_ij is a symbolic variable x_ij if (u_i, v_j) ∈ E and 0 otherwise, or

A^G_ij =





x_ij, if (u_i, v_j) ∈ E, 0, othersie.

Symbolic Determinants (continued)

• The matrix for the bipartite graph G on p. 481 is^a

A^G =













0 0 x₁₃ x₁₄ 0

0 x₂₂ 0 0 0

x₃₁ 0 0 0 x₃₅

x₄₁ 0 x₄₃ x₄₄ 0

x₅₁ 0 0 0 x₅₅













. (6)

aThe idea is similar to the Tanner graph in coding theory by Tanner (1981).

Symbolic Determinants (concluded)

• The determinant of A^G is det(A^G) = ∑

π

sgn(π)

∏n i=1

A^G_i,π(i). (7)

– π ranges over all permutations of n elements.

– sgn(π) is 1 if π is the product of an even number of transpositions and −1 otherwise.

– Equivalently, sgn(π) = 1 if the number of (i, j)s such that i < j and π(i) > π(j) is even.^a

• det(A^G) contains n! terms, many of which may be 0s.

aContributed by Mr. Hwan-Jeu Yu (D95922028) on May 1, 2008.

Determinant and Bipartite Perfect Matching

• In ∑

π sgn(π)∏n

i=1 A^G_i,π(i), note the following:

– Each summand corresponds to a possible perfect matching π.

– All of these summands ∏n

i=1 A^G_i,π(i) are distinct monomials and will not cancel.

• det(A^G) is essentially an exhaustive enumeration.

Proposition 58 (Edmonds (1967)) G has a perfect matching if and only if det(A^G) is not identically zero.

Perfect Matching and Determinant (p. 481)

X_

X_

X_

X_

X_

Y_

Y_

Y_

Y_

Y_

Perfect Matching and Determinant (concluded)

• The matrix is (p. 483)

A^G =













0 0 x₁₃ x₁₄ 0

0 x₂₂ 0 0 0

x₃₁ 0 0 0 x₃₅

x₄₁ 0 x₄₃ x₄₄ 0

x₅₁ 0 0 0 x₅₅











 .

• det(A^G) = −x¹⁴x₂₂x₃₅x₄₃x₅₁ + x₁₃x₂₂x₃₅x₄₄x₅₁ + x₁₄x₂₂x₃₁x₄₃x₅₅ − x¹³x₂₂x₃₁x₄₄x₅₅.

How To Test If a Polynomial Is Identically Zero?

• det(A^G) is a polynomial in n² variables.

• There are exponentially many terms in det(A^G).

• Expanding the determinant polynomial is not feasible.

– Too many terms.

• If det(A^G) ≡ 0, then it remains zero if we substitute arbitrary integers for the variables x₁₁, . . . , x_nn.

• When det(A^G) ̸≡ 0, what is the likelihood of obtaining a zero?

Number of Roots of a Polynomial

Lemma 59 (Schwartz (1980)) Let p(x₁, x₂, . . . , x_m) ̸≡ 0 be a polynomial in m variables each of degree at most d. Let M ∈ Z⁺. Then the number of m-tuples

(x₁, x₂, . . . , x_m) ∈ {0, 1, . . . , M − 1}^m such that p(x₁, x₂, . . . , x_m) = 0 is

≤ mdM^m−1.

• By induction on m (consult the textbook).

Density Attack

• The density of roots in the domain is at most mdM^m−1

M^m = md

M . (8)

• So suppose p(x¹, x₂, . . . , x_m) ̸≡ 0.

• Then a random

(x₁, x₂, . . . , x_m) ∈ { 0, 1, . . . , M − 1 }^m has a probability of ≤ md/M of being a root of p.

• Note that M is under our control!

– One can raise M to lower the error probability, e.g.

Density Attack (concluded)

Here is a sampling algorithm to test if p(x₁, x₂, . . . , x_m) ̸≡ 0.

1: Choose i₁, . . . , i_m from {0, 1, . . . , M − 1} randomly;

2: if p(i₁, i₂, . . . , i_m) ̸= 0 then

3: return “p is not identically zero”;

4: else

5: return “p is (probably) identically zero”;

6: end if

Analysis

• If p(x¹, x₂, . . . , x_m) ≡ 0 , the algorithm will always be correct as p(i₁, i₂, . . . , i_m) = 0.

• Suppose p(x¹, x₂, . . . , x_m) ̸≡ 0.

– The algorithm will answer incorrectly with

probability at most md/M by Eq. (8) on p. 490.

• We next return to the original problem of bipartite perfect matching.

A Randomized Bipartite Perfect Matching Algorithm

1: Choose n² integers i₁₁, . . . , i_nn from {0, 1, . . . , 2n² − 1}

randomly; {So M = 2n².}

2: Calculate det(A^G(i₁₁, . . . , i_nn)) by Gaussian elimination;

3: if det(A^G(i₁₁, . . . , i_nn)) ̸= 0 then

4: return “G has a perfect matching”;

5: else

6: return “G has no perfect matchings”;

7: end if

aLov´asz (1979). According to Paul Erd˝os, Lov´asz wrote his first sig- nificant paper “at the ripe old age of 17.”

Analysis

• If G has no perfect matchings, the algorithm will always be correct as det(A^G(i₁₁, . . . , i_nn)) = 0.

• Suppose G has a perfect matching.

– The algorithm will answer incorrectly with

probability at most md/M = 0.5 with m = n², d = 1 and M = 2n² in Eq. (8) on p. 490.

• Run the algorithm independently k times.

• Output “G has no perfect matchings” if and only if all say no.

• The error probability is now reduced to at most 2^−k.

L´ oszl´ o Lov´ asz (1948–)

Remarks

• Note that we are calculating

prob[ algorithm answers “no”| G has no perfect matchings ], prob[ algorithm answers “yes”| G has a perfect matching ].

• We are not calculating^b

prob[ G has no perfect matchings| algorithm answers “no” ], prob[ G has a perfect matching| algorithm answers “yes” ].

aThanks to a lively class discussion on May 1, 2008.

bNumerical Recipes in C (1988), “[As] we already remarked, statistics is not a branch of mathematics!”

But How Large Can det(A

(i

, . . . , i

)) Be?

• It is at most

n! (

2n²)n

.

• Stirling’s formula says n! ∼ √

2πn (n/e)ⁿ.

• Hence

log₂ det(A^G(i₁₁, . . . , i_nn)) = O(n log₂ n) bits are sufficient for representing the determinant.

• We skip the details about how to make sure that all

An Intriguing Question

• Is there an (i¹¹, . . . , i_nn) that will always give correct answers for the algorithm on p. 493?

• A theorem on p. 591 shows that such an (i¹¹, . . . , i_nn) exists!

– Whether it can be found efficiently is another matter.

• Once (i¹¹, . . . , i_nn) is available, the algorithm can be made deterministic.

aThanks to a lively class discussion on November 24, 2004.

Randomization vs. Nondeterminism

• What are the differences between randomized algorithms and nondeterministic algorithms?

• One can think of a randomized algorithm as a

nondeterministic algorithm but with a probability associated with every guess/branch.

• So each computation path of a randomized algorithm has a probability associated with it.

aContributed by Mr. Olivier Valery (D01922033) and Mr. Hasan Al- hasan (D01922034) on November 27, 2012.

Monte Carlo Algorithms

• The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that

– If the algorithm finds that a matching exists, it is always correct (no false positives).

– If the algorithm answers in the negative, then it may make an error (false negatives).

aMetropolis and Ulam (1949).

Monte Carlo Algorithms (continued)

• The algorithm makes a false negative with probability

≤ 0.5.^a

– Note this probability refers to^b

prob[ algorithm answers “no”| G has a perfect matching ] not

prob[ G has a perfect matching| algorithm answers “no” ].

aEquivalently, among the coin flip sequences, at most half of them lead to the wrong answer.

| input is a “yes” instance ].

Monte Carlo Algorithms (concluded)

• This probability o.5 is not over the space of all graphs or determinants, but over the algorithm’s own coin flips.

– It holds for any bipartite graph.

The Markov Inequality

Lemma 60 Let x be a random variable taking nonnegative integer values. Then for any k > 0,

prob[ x ≥ kE[ x ] ] ≤ 1/k.

• Let pⁱ denote the probability that x = i.

E[ x ] = ∑

i

ip_i = ∑

i<kE[ x ]

ip_i + ∑

i≥kE[ x ]

ip_i

≥ ∑

i≥kE[ x ]

ip_i ≥ kE[ x ] ∑

i≥kE[ x ]

p_i

≥ kE[ x ] × prob[x ≥ kE[ x ]].

Andrei Andreyevich Markov (1856–1922)

An Application of Markov’s Inequality

• Suppose algorithm C runs in expected time T (n) and always gives the right answer.

• Consider an algorithm that runs C for time kT (n) and rejects the input if C does not stop within the time bound.

• By Markov’s inequality, this new algorithm runs in time kT (n) and gives the wrong answer with probability

≤ 1/k.

An Application of Markov’s Inequality (concluded)

• By running this algorithm m times (the total running time is mkT (n)), we reduce the error probability to

≤ k^−m.^a

• Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound.

• By Markov’s inequality, this new algorithm gives the wrong answer with probability ≤ 1/(mk).

• This is much worse than the previous algorithm’s error probability of ≤ k^−m for the same amount of time.

aWith the same input. Thanks to a question on December 7, 2010.

fsat for k-sat Formulas (p. 469)

• Let ϕ(x¹, x₂, . . . , x_n) be a k-sat formula.

• If ϕ is satisfiable, then return a satisfying truth assignment.

• Otherwise, return “no.”

• We next propose a randomized algorithm for this problem.

A Random Walk Algorithm for ϕ in CNF Form

1: Start with an arbitrary truth assignment T ;

2: for i = 1, 2, . . . , r do

3: if T |= ϕ then

4: return “ϕ is satisfiable with T ”;

5: else

6: Let c be an unsatisfied clause in ϕ under T ; {All of its literals are false under T .}

7: Pick any x of these literals at random;

8: Modify T to make x true;

9: end if

10: end for

11: return “ϕ is unsatisfiable”;

3sat vs. 2sat Again

• Note that if ϕ is unsatisfiable, the algorithm will not refute it.

• The random walk algorithm needs expected exponential time for 3sat.

– In fact, it runs in expected O((1.333· · · + ϵ)ⁿ) time with r = 3n,^a much better than O(2ⁿ).^b

• We will show immediately that it works well for 2sat.

• The state of the art as of 2006 is expected O(1.322ⁿ) time for 3sat and expected O(1.474ⁿ) time for 4sat.^c

aUse this setting per run of the algorithm.

Random Walk Works for 2sat

Theorem 61 Suppose the random walk algorithm with r = 2n² is applied to any satisfiable 2sat problem with n variables. Then a satisfying truth assignment will be

discovered with probability at least 0.5.

• Let ˆT be a truth assignment such that ˆT |= ϕ.

• Assume our starting T differs from ˆT in i values.

– Their Hamming distance is i.

– Recall T is arbitrary.

aPapadimitriou (1991).

The Proof

• Let t(i) denote the expected number of repetitions of the flipping step^a until a satisfying truth assignment is

found.

• It can be shown that t(i) is finite.

• t(0) = 0 because it means that T = ˆT and hence T |= ϕ.

• If T ̸= ˆT or any other satisfying truth assignment, then we need to flip the coin at least once.

• We flip a coin to pick among the 2 literals of a clause not satisfied by the present T .

• At least one of the 2 literals is true under ˆT because ˆT

The Proof (continued)

• So we have at least 0.5 chance of moving closer to ˆT .

• Thus

t(i) ≤ t(i − 1) + t(i + 1)

2 + 1

for 0 < i < n.

– Inequality is used because, for example, T may differ from ˆT in both literals.

• It must also hold that

t(n) ≤ t(n − 1) + 1 because at i = n, we can only decrease i.

The Proof (continued)

• Now, put the necessary relations together:

t(0) = 0, (9)

t(i) ≤ t(i − 1) + t(i + 1)

2 + 1, 0 < i < n, (10)

t(n) ≤ t(n − 1) + 1. (11)

• Technically, this is a one-dimensional random walk with an absorbing barrier at i = 0 and a reflecting barrier at i = n (if we replace “≤” with “=”).^a

aThe proof in the textbook does exactly that. But a student pointed

The Proof (continued)

• Add up the relations for

2t(1), 2t(2), 2t(3), . . . , 2t(n − 1), t(n) to obtain^a 2t(1) + 2t(2) + · · · + 2t(n − 1) + t(n)

≤ t(0) + t(1) + 2t(2) + · · · + 2t(n − 2) + 2t(n − 1) + t(n) +2(n − 1) + 1.

• Simplify it to yield

t(1) ≤ 2n − 1. (12)

aAdding up the relations for t(1), t(2), t(3), . . . , t(n−1) will also work, thanks to Mr. Yen-Wu Ti (D91922010).

The Proof (continued)

• Add up the relations for 2t(2), 2t(3), . . . , 2t(n − 1), t(n) to obtain

2t(2) + · · · + 2t(n − 1) + t(n)

≤ t(1) + t(2) + 2t(3) + · · · + 2t(n − 2) + 2t(n − 1) + t(n) +2(n − 2) + 1.

• Simplify it to yield

t(2) ≤ t(1) + 2n − 3 ≤ 2n − 1 + 2n − 3 = 4n − 4

The Proof (continued)

• Continuing the process, we shall obtain t(i) ≤ 2in − i².

• The worst upper bound happens when i = n, in which case

t(n) ≤ n².

• We conclude that

t(i) ≤ t(n) ≤ n² for 0 ≤ i ≤ n.

The Proof (concluded)

• So the expected number of steps is at most n².

• The algorithm picks r = 2n².

– This amounts to invoking the Markov inequality

(p. 503) with k = 2, resulting in a probability of 0.5.^a

• The proof does not yield a polynomial bound for 3sat.^b

aRecall p. 505.

bContributed by Mr. Cheng-Yu Lee (R95922035) on November 8, 2006.

Christos Papadimitriou (1949–)

Boosting the Performance

• We can pick r = 2mn² to have an error probability of

≤ 1 2m by Markov’s inequality.

• Alternatively, with the same running time, we can run the “r = 2n²” algorithm m times.

• The error probability is now reduced to

≤ 2^−m.

Primality Tests

• primes asks if a number N is a prime.

• The classic algorithm tests if k | N for k = 2, 3, . . . ,√ N .

• But it runs in Ω(2^{(log2 N )/2}) steps.

Primality Tests (concluded)

• Suppose N = P Q is a product of 2 distinct primes.

• The probability of success of the density attack (p. 450) is

≈ 2

√N when P ≈ Q.

• This probability is exponentially small in terms of the input length log₂ N .

The Fermat Test for Primality

Fermat’s “little” theorem (p. 453) suggests the following primality test for any given number N :

1: Pick a number a randomly from {1, 2, . . . , N − 1};

2: if a^N−1 ̸= 1 mod N then

3: return “N is composite”;

4: else

5: return “N is a prime”;

6: end if

The Fermat Test for Primality (concluded)

• Carmichael numbers are composite numbers that will pass the Fermat test for all a ∈ {1, 2, . . . , N − 1}.^a

– The Fermat test will return “N is a prime” for all Carmichael numbers N .

• Unfortunately, there are infinitely many Carmichael numbers.^b

• In fact, the number of Carmichael numbers less than N exceeds N^2/7 for N large enough.

• So the Fermat test is an incorrect algorithm for primes.

aCarmichael (1910). Lo (1994) mantions an investment strategy

Square Roots Modulo a Prime

• Equation x² = a mod p has at most two (distinct) roots by Lemma 57 (p. 458).

– The roots are called square roots.

– Numbers a with square roots and gcd(a, p) = 1 are called quadratic residues.

∗ They are

1² mod p, 2² mod p, . . . , (p − 1)² mod p.

• We shall show that a number either has two roots or has none, and testing which is the case is trivial.^a

aBut no efficient deterministic general-purpose square-root-extracting algorithms are known yet.

Euler’s Test

Lemma 62 (Euler) Let p be an odd prime and a ̸= 0 mod p.

1. If

a^(p−1)/2 = 1 mod p, then x² = a mod p has two roots.

2. If

a^(p−1)/2 ̸= 1 mod p, then

a^(p−1)/2 = −1 mod p

The Proof (continued)

• Let r be a primitive root of p.

• By Fermat’s “little” theorem, r^(p−1)/2 is a square root of 1.

• So

r^(p−1)/2 = 1 or −1 mod p.

• But as r is a primitive root, r^(p−1)/2 ̸= 1 mod p.

• Hence

r^(p−1)/2 = −1 mod p.

The Proof (continued)

• Let a = r^k mod p for some k.

• Then

1 = a^(p−1)/2 = r^k(p−1)/2 = [

r^(p−1)/2 ]k

= (−1)^k mod p.

• So k must be even.

• Suppose a = r^2j for some 1 ≤ j ≤ (p − 1)/2.

• Then a^(p−1)/2 = r^j(p−1) = 1 mod p, and a’s two distinct roots are r^j,−r^j(= r^j+(p−1)/2 mod p).

– If r^j = −r^j mod p, then 2r^j = 0 mod p, which implies

The Proof (continued)

• As 1 ≤ j ≤ (p − 1)/2, there are (p − 1)/2 such a’s.

• Each such a has 2 distinct square roots.

• The square roots of all the a’s are distinct.

– The square roots of different a’s must be different.

• Hence the set of square roots is {1, 2, . . . , p − 1}.

• As a result, a = r^2j, 1 ≤ j ≤ (p − 1)/2, exhaust all the quadratic residues.

The Proof (concluded)

• If a = r^2j+1, then it has no roots because all the square roots have been taken.

• Now,

a^(p−1)/2 = [

r^(p−1)/2

]2j+1

= (−1)^2j+1 = −1 mod p.

The Legendre Symbol^a and Quadratic Residuacity Test

• By Lemma 62 (p. 525) a^(p−1)/2 mod p = ±1 for a ̸= 0 mod p.

• For odd prime p, define the Legendre symbol (a | p) as

(a| p) =









0 if p| a,

1 if a is a quadratic residue modulo p,

−1 if a is a quadratic nonresidue modulo p.

• Euler’s test (p. 525) implies

a^(p−1)/2 = (a| p) mod p for any odd prime p and any integer a.

• Note that (ab|p) = (a|p)(b|p).