• 沒有找到結果。

# Monte Carlo Algorithms

N/A
N/A
Protected

Share "Monte Carlo Algorithms"

Copied!
49
0
0

(1)

## Monte Carlo Algorithms

a

The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that

– If the algorithm finds that a matching exists, it is always correct (no false positives).

– If the algorithm answers in the negative, then it may make an error (false negative).

aMetropolis and Ulam (1949).

(2)

## Monte Carlo Algorithms (concluded)

The algorithm makes a false negative with probability

≤ 0.5.

– Note this probability refers to

prob[ algorithm answers “no” | G has a perfect matching ] not

prob[ G has a perfect matching | algorithm answers “no” ].

This probability is not over the space of all graphs or determinants, but over the algorithm’s own coin flips.

It holds for any bipartite graph.

(3)

## False Positives and False Negatives in Human Behavior?

a

“[Men] tend to misinterpret innocent friendliness as a sign that women are [· · · ] interested in them.”

– A false positive.

“[Women] tend to undervalue signs that a man is interested in a committed relationship.”

– A false negative.

a“Don’t misunderestimate yourself.” The Economist, 2006.

(4)

## The Markov Inequality

a

Lemma 64 Let x be a random variable taking nonnegative integer values. Then for any k > 0,

prob[x ≥ kE[ x ]] ≤ 1/k.

Let pi denote the probability that x = i.

E[ x ] = X

i

ipi

= X

i<kE[ x ]

ipi + X

i≥kE[ x ]

ipi

≥ kE[ x ] × prob[x ≥ kE[ x ]].

aAndrei Andreyevich Markov (1856–1922).

(5)

(6)

## An Application of Markov’s Inequality

Algorithm C runs in expected time T (n) and always gives the right answer.

Consider an algorithm that runs C for time kT (n) and rejects the input if C does not stop within the time bound.

By Markov’s inequality, this new algorithm runs in time kT (n) and gives the wrong answer with probability

≤ 1/k.

By running this algorithm m times, we reduce the error probability to ≤ k−m.

(7)

## An Application of Markov’s Inequality (concluded)

Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound.

By Markov’s inequality, this new algorithm gives the wrong answer with probability ≤ 1/(mk).

This is a far cry from the previous algorithm’s error probability of ≤ k−m.

(8)

## fsat for k-sat Formulas (p. 427)

Let φ(x1, x2, . . . , xn) be a k-sat formula.

If φ is satisfiable, then return a satisfying truth assignment.

Otherwise, return “no.”

We next propose a randomized algorithm for this problem.

(9)

## A Random Walk Algorithm for φ in CNF Form

1: Start with an arbitrary truth assignment T ;

2: for i = 1, 2, . . . , r do

3: if T |= φ then

4: return “φ is satisfiable with T ”;

5: else

6: Let c be an unsatisfiable clause in φ under T ; {All of its literals are false under T .}

7: Pick any x of these literals at random;

8: Modify T to make x true;

9: end if

10: end for

11: return “φ is unsatisfiable”;

(10)

## 3sat vs. 2sat Again

Note that if φ is unsatisfiable, the algorithm will not refute it.

The random walk algorithm needs expected exponential time for 3sat.

In fact, it runs in expected O((1.333 · · · + ²)n) time with r = 3n,a much better than O(2n).b

We will show immediately that it works well for 2sat.

The state of the art as of 2006 is expected O(1.322n) time for 3sat and expected O(1.474n) time for 4sat.c

aUse this setting per run of the algorithm.

bSch¨oning (1999).

(11)

## Random Walk Works for 2sat

a

Theorem 65 Suppose the random walk algorithm with r = 2n2 is applied to any satisfiable 2sat problem with n variables. Then a satisfying truth assignment will be

discovered with probability at least 0.5.

Let ˆT be a truth assignment such that ˆT |= φ.

Let t(i) denote the expected number of repetitions of the flipping step until a satisfying truth assignment is found if our starting T differs from ˆT in i values.

Their Hamming distance is i.

Recall T is arbitrary.

(12)

## The Proof

It can be shown that t(i) is finite.

t(0) = 0 because it means that T = ˆT and hence T |= φ.

If T 6= ˆT or T is not equal to any other satisfying truth assignment, then we need to flip at least once.

We flip to pick among the 2 literals of a clause not satisfied by the present T .

At least one of the 2 literals is true under ˆT because ˆT satisfies all clauses.

So we have at least 0.5 chance of moving closer to ˆT .

(13)

## The Proof (continued)

Thus

t(i) ≤ t(i − 1) + t(i + 1)

2 + 1

for 0 < i < n.

Inequality is used because, for example, T may differ from ˆT in both literals.

It must also hold that

t(n) ≤ t(n − 1) + 1 because at i = n, we can only decrease i.

(14)

## The Proof (continued)

As we are only interested in upper bounds, we solve x(0) = 0

x(n) = x(n − 1) + 1

x(i) = x(i − 1) + x(i + 1)

2 + 1, 0 < i < n

This is one-dimensional random walk with a reflecting and an absorbing barrier.

(15)

## The Proof (continued)

Add the equations up to obtain

x(1) + x(2) + · · · + x(n)

= x(0) + x(1) + 2x(2) + · · · + 2x(n − 2) + x(n − 1) + x(n) 2

+n + x(n − 1).

Simplify to yield

x(1) + x(n) − x(n − 1)

2 = n.

As x(n) − x(n − 1) = 1, we have x(1) = 2n − 1.

(16)

## The Proof (continued)

Iteratively, we obtain

x(2) = 4n − 4, ...

x(i) = 2in − i2.

The worst case happens when i = n, in which case x(n) = n2.

(17)

## The Proof (concluded)

We therefore reach the conclusion that t(i) ≤ x(i) ≤ x(n) = n2.

So the expected number of steps is at most n2.

The algorithm picks a running time 2n2.

This amounts to invoking the Markov inequality (p. 462) with k = 2, with the consequence of having a probability of 0.5.

The proof does not yield a polynomial bound for 3sat.a

aContributed by Mr. Cheng-Yu Lee (R95922035) on November 8, 2006.

(18)

## Boosting the Performance

We can pick r = 2mn2 to have an error probability of

≤ (2m)−1 by Markov’s inequality.

Alternatively, with the same running time, we can run the “r = 2n2” algorithm m times.

But the error probability is reduced to ≤ 2−m!

(19)

## Primality Tests

primes asks if a number N is a prime.

The classic algorithm tests if k | N for k = 2, 3, . . . ,√ N .

But it runs in Ω(2n/2) steps, where n = | N | = log2 N .

(20)

## The Density Attack for primes

1: Pick k ∈ {2, . . . , N − 1} randomly; {Assume N > 2.}

2: if k | N then

3: return “N is composite”;

4: else

5: return “N is a prime”;

6: end if

(21)

## Analysis

a

Suppose N = P Q, a product of 2 primes.

The probability of success is

< 1 − φ(N )

N = 1 − (P − 1)(Q − 1)

P Q = P + Q − 1 P Q .

In the case where P ≈ Q, this probability becomes

< 1

P + 1

Q 2

√N .

This probability is exponentially small.

(22)

## The Fermat Test for Primality

Fermat’s “little” theorem on p. 411 suggests the following primality test for any given number p:

1: Pick a number a randomly from {1, 2, . . . , N − 1};

2: if aN −1 6= 1 mod N then

3: return “N is composite”;

4: else

5: return “N is a prime”;

6: end if

(23)

## The Fermat Test for Primality (concluded)

Unfortunately, there are composite numbers called Carmichael numbers that will pass the Fermat test for all a ∈ {1, 2, . . . , N − 1}.a

There are infinitely many Carmichael numbers.b

In fact, the number of Carmichael numbers less than n exceeds n2/7 for n large enough.

aCarmichael (1910).

bAlford, Granville, and Pomerance (1992).

(24)

## Square Roots Modulo a Prime

Equation x2 = a mod p has at most two (distinct) roots by Lemma 59 (p. 416).

– The roots are called square roots.

Numbers a with square roots and gcd(a, p) = 1 are called quadratic residues.

They are 12 mod p, 22 mod p, . . . , (p − 1)2 mod p.

We shall show that a number either has two roots or has none, and testing which one is true is trivial.

There are no known efficient deterministic algorithms to find the roots, however.

(25)

## Euler’s Test

Lemma 66 (Euler) Let p be an odd prime and a 6= 0 mod p.

1. If a(p−1)/2 = 1 mod p, then x2 = a mod p has two roots.

2. If a(p−1)/2 6= 1 mod p, then a(p−1)/2 = −1 mod p and x2 = a mod p has no roots.

Let r be a primitive root of p.

By Fermat’s “little” theorem, r(p−1)/2 is a square root of 1, so r(p−1)/2 = 1 mod p or r(p−1)/2 = −1 mod p.

But as r is a primitive root, r(p−1)/2 6= 1 mod p.

Hence r(p−1)/2 = −1 mod p.

(26)

## The Proof (continued)

Let a = rk mod p for some k.

Then

1 = a(p−1)/2 = rk(p−1)/2 = h

r(p−1)/2 ik

= (−1)k mod p.

So k must be even.

Suppose a = r2j for some 1 ≤ j ≤ (p − 1)/2.

Then a(p−1)/2 = rj(p−1) = 1 mod p and its two distinct roots are rj, −rj(= rj+(p−1)/2 mod p).

If rj = −rj mod p, then 2rj = 0 mod p, which implies rj = 0 mod p, a contradiction.

(27)

## The Proof (continued)

As 1 ≤ j ≤ (p − 1)/2, there are (p − 1)/2 such a’s.

Each such a has 2 distinct square roots.

The square roots of all the a’s are distinct.

The square roots of different a’s must be different.

Hence the set of square roots is {1, 2, . . . , p − 1}.

Because there are (p − 1)/2 such a’s and each a has two square roots.

As a result, a = r2j, 1 ≤ j ≤ (p − 1)/2, exhaust all the quadratic residues.

(28)

## The Proof (concluded)

If a = r2j+1, then it has no roots because all the square roots have been taken.

Now,

a(p−1)/2 = h

r(p−1)/2

i2j+1

= (−1)2j+1 = −1 mod p.

(29)

The Legendre Symbola and Quadratic Residuacity Test

By Lemma 66 (p. 483) a(p−1)/2 mod p = ±1 for a 6= 0 mod p.

For odd prime p, define the Legendre symbol (a | p) as

(a | p) =

0 if p | a,

1 if a is a quadratic residue modulo p,

−1 if a is a quadratic nonresidue modulo p.

Euler’s test implies a(p−1)/2 = (a | p) mod p for any odd prime p and any integer a.

Note that (ab|p) = (a|p)(b|p).

aAndrien-Marie Legendre (1752–1833).

(30)

## Gauss’s Lemma

Lemma 67 (Gauss) Let p and q be two odd primes. Then (q|p) = (−1)m, where m is the number of residues in

R = { iq mod p : 1 ≤ i ≤ (p − 1)/2 } that are greater than (p − 1)/2.

All residues in R are distinct.

If iq = jq mod p, then p|(j − i) q or p|q.

No two elements of R add up to p.

If iq + jq = 0 mod p, then p|(i + j) or p|q.

– But neither is possible.

(31)

## The Proof (continued)

Consider the set R0 of residues that result from R if we replace each of the m elements a ∈ R such that

a > (p − 1)/2 by p − a.

This is equivalent to performing −a mod p.

All residues in R0 are now at most (p − 1)/2.

In fact, R0 = {1, 2, . . . , (p − 1)/2} (see illustration next page).

Otherwise, two elements of R would add up to p, which has been shown to be impossible.

(32)

5 1 2 3 4

6 5

1 2 3 4

6

p = 7 and q = 5.

(33)

## The Proof (concluded)

Alternatively, R0 = {±iq mod p : 1 ≤ i ≤ (p − 1)/2}, where exactly m of the elements have the minus sign.

Take the product of all elements in the two representations of R0.

So [(p − 1)/2]! = (−1)mq(p−1)/2[(p − 1)/2]! mod p.

Because gcd([(p − 1)/2]!, p) = 1, the above implies 1 = (−1)mq(p−1)/2 mod p.

(34)

## Legendre’s Law of Quadratic Reciprocity

a

Let p and q be two odd primes.

The next result says their Legendre symbols are distinct if and only if both numbers are 3 mod 4.

Lemma 68 (Legendre (1785), Gauss)

(p|q)(q|p) = (−1)p−12 q−12 .

aFirst stated by Euler in 1751. Legendre (1785) did not give a correct proof. Gauss proved the theorem when he was 19. He gave at least 6 different proofs during his life. The 152nd proof appeared in 1963.

(35)

## The Proof (continued)

Sum the elements of R0 in the previous proof in mod2.

On one hand, this is just P(p−1)/2

i=1 i mod 2.

On the other hand, the sum equals

(p−1)/2X

i=1

µ

qi − p

¹iq p

º¶

+ mp mod 2

=

q

(p−1)/2X

i=1

i − p

(p−1)/2X

i=1

¹iq p

º

 + mp mod 2.

– Signs are irrelevant under mod2.

m is as in Lemma 67 (p. 488).

(36)

## The Proof (continued)

Ignore odd multipliers to make the sum equal

(p−1)/2X

i=1

i −

(p−1)/2X

i=1

¹iq p

º

 + m mod 2.

Equate the above with P(p−1)/2

i=1 i mod 2 to obtain

m =

(p−1)/2X

i=1

¹iq p

º

mod 2.

(37)

## The Proof (concluded)

P(p−1)/2

i=1 biqp c is the number of integral points under the line y = (q/p) x for 1 ≤ x ≤ (p − 1)/2.

Gauss’s lemma (p. 488) says (q|p) = (−1)m.

Repeat the proof with p and q reversed.

So (p|q) = (−1)m0, where m0 is the number of integral points above the line y = (q/p) x for 1 ≤ y ≤ (q − 1)/2.

As a result, (p|q)(q|p) = (−1)m+m0.

But m + m0 is the total number of integral points in the

p−1

2 × q−12 rectangle, which is p−12 q−12 .

(38)

## Eisenstein’s Rectangle

(p,q)

(p - 1)/2 (q - 1)/2

p = 11 and q = 7.

(39)

## The Jacobi Symbol

a

The Legendre symbol only works for odd prime moduli.

The Jacobi symbol (a | m) extends it to cases where m is not prime.

Let m = p1p2 · · · pk be the prime factorization of m.

When m > 1 is odd and gcd(a, m) = 1, then

(a|m) = Yk i=1

(a | pi).

Note that the Jacobi symbol equals ±1.

It reduces to the Legendre symbol when m is a prime.

Define (a | 1) = 1.

(40)

## Properties of the Jacobi Symbol

The Jacobi symbol has the following properties, for arguments for which it is defined.

1. (ab | m) = (a | m)(b | m).

2. (a | m1m2) = (a | m1)(a | m2).

3. If a = b mod m, then (a | m) = (b | m).

4. (−1 | m) = (−1)(m−1)/2 (by Lemma 67 on p. 488).

5. (2 | m) = (−1)(m2−1)/8.a

6. If a and m are both odd, then (a | m)(m | a) = (−1)(a−1)(m−1)/4.

(41)

## Calculation of (2200|999)

Similar to the Euclidean algorithm and does not require factorization.

(202|999) = (−1)(9992−1)/8(101|999)

= (−1)124750(101|999) = (101|999)

= (−1)(100)(998)/4(999|101) = (−1)24950(999|101)

= (999|101) = (90|101) = (−1)(1012−1)/8(45|101)

= (−1)1275(45|101) = −(45|101)

= −(−1)(44)(100)/4(101|45) = −(101|45) = −(11|45)

= −(−1)(10)(44)/4(45|11) = −(45|11)

= −(1|11) = −1.

(42)

## A Result Generalizing Proposition 10.3 in the Textbook

Theorem 69 The group of set Φ(n) under multiplication mod n has a primitive root if and only if n is either 1, 2, 4, pk, or 2pk for some nonnegative integer k and and odd

prime p.

This result is essential in the proof of the next lemma.

(43)

## The Jacobi Symbol and Primality Test

a

Lemma 70 If (M |N ) = M(N −1)/2 mod N for all M ∈ Φ(N ), then N is prime. (Assume N is odd.)

Assume N = mp, where p is an odd prime, gcd(m, p) = 1, and m > 1 (not necessarily prime).

Let r ∈ Φ(p) such that (r | p) = −1.

The Chinese remainder theorem says that there is an M ∈ Φ(N ) such that

M = r mod p, M = 1 mod m.

aMr. Clement Hsiao (R88526067) pointed out that the textbook’s proof for Lemma 11.8 is incorrect while he was a senior in January 1999.

(44)

## The Proof (continued)

By the hypothesis,

M(N −1)/2 = (M | N ) = (M | p)(M | m) = −1 mod N.

Hence

M(N −1)/2 = −1 mod m.

But because M = 1 mod m,

M(N −1)/2 = 1 mod m, a contradiction.

(45)

## The Proof (continued)

Second, assume that N = pa, where p is an odd prime and a ≥ 2.

By Theorem 69 (p. 500), there exists a primitive root r modulo pa.

From the assumption, MN −1 =

h

M(N −1)/2 i2

= (M |N )2 = 1 mod N for all M ∈ Φ(N ).

(46)

## The Proof (continued)

As r ∈ Φ(N ) (prove it), we have

rN −1 = 1 mod N.

As r’s exponent modulo N = pa is φ(N ) = pa−1(p − 1), pa−1(p − 1) | N − 1,

which implies that p | N − 1.

But this is impossible given that p | N .

(47)

## The Proof (continued)

Third, assume that N = mpa, where p is an odd prime, gcd(m, p) = 1, m > 1 (not necessarily prime), and a is even.

The proof mimics that of the second case.

By Theorem 69 (p. 500), there exists a primitive root r modulo pa.

From the assumption, MN −1 =

h

M(N −1)/2 i2

= (M |N )2 = 1 mod N for all M ∈ Φ(N ).

(48)

## The Proof (continued)

In particular,

MN −1 = 1 mod pa (7)

for all M ∈ Φ(N ).

The Chinese remainder theorem says that there is an M ∈ Φ(N ) such that

M = r mod pa, M = 1 mod m.

Because M = r mod pa and Eq. (7), rN −1 = 1 mod pa.

(49)

## The Proof (concluded)

As r’s exponent modulo N = pa is φ(N ) = pa−1(p − 1), pa−1(p − 1) | N − 1,

which implies that p | N − 1.

But this is impossible given that p | N .

This occurs when tooth eruption does not occur within a normal time frame, the tooth is not present in the dental arch, and there is no potential for eruption due to the presence of

• A concrete problem is polynomial-time solvable if there exists an algorithm that solves any concrete instance of length n in time for some constant k. • Solvable = can produce

Using

• The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that. – If the algorithm ﬁnds that a matching exists, it is always correct (no

Previous efficient algorithm, Forward Algorithm, needs O(m 3/2 ) time and O(m) space. To develop algorithms on random access machines, we come up with

• Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound. • By Markov’s inequality, this

• Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound.. • By Markov’s inequality, this

• An algorithm for such a problem whose running time is a polynomial of the input length and the value (not length) of the largest integer parameter is a..

• The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that.. – If the algorithm ﬁnds that a matching exists, it is always correct

• Consider an algorithm that runs C for time kT (n) and rejects the input if C does not stop within the time bound.. • By Markov’s inequality, this new algorithm runs in time kT (n)

• By definition, a pseudo-polynomial-time algorithm becomes polynomial-time if each integer parameter is limited to having a value polynomial in the input length.. • Corollary 42

• Consider an algorithm that runs C for time kT (n) and rejects the input if C does not stop within the time bound.. • By Markov’s inequality, this new algorithm runs in time kT (n)

• The randomized bipartite perfect matching algorithm is called a Monte Carlo algorithm in the sense that.. – If the algorithm ﬁnds that a matching exists, it is always correct

• Suppose, instead, we run the algorithm for the same running time mkT (n) once and rejects the input if it does not stop within the time bound.. • By Markov’s inequality, this

• The .NET Framework provides a run-time environment called the common language runtime, which runs the code and provides services that make the development process easier. •

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued).. • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued).. • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −

Breu and Kirk- patrick [35] (see [4]) improved this by giving O(nm 2 )-time algorithms for the domination and the total domination problems and an O(n 2.376 )-time algorithm for

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued).. • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued). • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued).. • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued).. • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −

Biases in Pricing Continuously Monitored Options with Monte Carlo (continued).. • If all of the sampled prices are below the barrier, this sample path pays max(S(t n ) −