Algorithm Design and Analysis Divide and Conquer (2)

Algorithm Design and Analysis Divide and Conquer (2)

http://ada.miulab.tw

Outline

• Recurrence (遞迴)

• Divide-and-Conquer

• D&C #1: Tower of Hanoi (河內塔)

• D&C #2: Merge Sort

• D&C #3: Bitonic Champion

• D&C #4: Maximum Subarray

• Solving Recurrences

• Substitution Method

• Recursion-Tree Method

• Master Method

• D&C #5: Matrix Multiplication

• D&C #6: Selection Problem

• D&C #7: Closest Pair of Points Problem

Divide-and-Conquer 之神乎奇技

Divide-and-Conquer 首部曲

What is Divide-and-Conquer?

• Solve a problem recursively

• Apply three steps at each level of the recursion

1. Divide the problem into a number of subproblems that are smaller instances of the

2. Conquer the subproblems by solving them recursively

• then solve the subproblems

• else recursively solve itself

3. Combine

base case recursive case

Solving Recurrences

Textbook Chapter 4.3 – The substitution method for solving recurrences Textbook Chapter 4.4 – The recursion-tree method for solving recurrences Textbook Chapter 4.5 – The master method for solving recurrences

D&C Algorithm Time Complexity

• 𝑇 𝑛 : running time for input size 𝑛

• 𝐷 𝑛 : time of Divide for input size 𝑛

• 𝐶 𝑛 : time of Combine for input size 𝑛

• 𝑎: number of subproblems

• 𝑛/𝑏: size of each subproblem

Solving Recurrences

1. Substitution Method (取代法)

• Guess a bound and then prove by induction

2. Recursion-Tree Method (遞迴樹法)

• Expand the recurrence into a tree and sum up the cost

3. Master Method (套公式大法/大師法)

• Apply Master Theorem to a specific form of recurrences

• Useful simplification tricks

• Ignore floors, ceilings, boundary conditions (proof in Ch. 4.6)

• Assume base cases are constant (for small n)

Substitution Method

Textbook Chapter 4.3 – The substitution method for solving recurrences

Review

• Time Complexity for Merge Sort

• Theorem

• Proof

• There exists positive constant 𝑎, 𝑏 s.t.

• Use induction to prove

• n = 1, trivial

• n > 1,

Substitution Method (取代法)

guess a bound and then prove by induction

Substitution Method (取代法)

• Guess the form of the solution

• Verify by mathematical induction (數學歸納法)

• Prove it works for 𝑛 = 1

• Prove that if it works for 𝑛 = 𝑚, then it works for 𝑛 = 𝑚 + 1

→ It can work for all positive integer 𝑛

• Solve constants to show that the solution works

• Prove 𝑂 and Ω separately

1. Guess

2. Verify

3. Solve

Substitution Method Example

• Proof

•

There exists positive constants 𝑛

, 𝑐 s.t. for all 𝑛 ≥ 𝑛

,

• Use induction to find the constants 𝑛

, 𝑐

• n = 1, trivial

• n > 1,

• holds when

e.g.

Inductive hypothesis

Guess

Verify

Solve

Substitution Method Example

• Proof

•

There exists positive constants 𝑛

, 𝑐 s.t. for all 𝑛 ≥ 𝑛

,

• Use induction to find the constants 𝑛

, 𝑐

• n = 1, trivial

• n > 1,

Inductive hypothesis

Tighter upper bound?

証不出來…

猜錯了？還是推導錯了？

沒猜錯 推導也沒錯 這是取代法的小盲點

Substitution Method Example

• Proof

•

There exists positive constants 𝑛

, 𝑐

, 𝑐

s.t. for all 𝑛 ≥ 𝑛

,

• Use induction to find the constants 𝑛

,𝑐

, 𝑐

• n = 1, holds for

• n > 1,

• holds when

e.g.

Inductive hypothesis

Guess

Verify

Solve Strengthen the inductive hypothesis

by subtracting a low-order term

Useful Tricks

• Guess based on seen recurrences

• Use the recursion-tree method

• From loose bound to tight bound

• Strengthen the inductive hypothesis by subtracting a low-order term

• Change variables

• E.g.,

1. Change variable:

2. Change variable again:

3. Solve recurrence

Recursion-Tree Method

Textbook Chapter 4.4 – The recursion-tree method for solving recurrences

Review

• Time Complexity for Merge Sort

• Theorem

• Proof

2^nd expansion 1^st expansion

k^th expansion

The expansion stops when 2^𝑘 = 𝑛

Recursion-Tree Method (遞迴樹法)

Expand the recurrence into a tree and sum up the cost

Recursion-Tree Method (遞迴樹法)

• Expand a recurrence into a tree

• Sum up the cost of all nodes as a good guess

• Verify the guess as in the substitution method

• Advantages

• Promote intuition

• Generate good guesses for the substitution method

1. Expand

2. Sumup

3. Verify

Recursion-Tree Example

Recursion-Tree Example

Recursion-Tree Example

Recursion-Tree Example

+

Master Theorem

Textbook Chapter 4.4 – The recursion-tree method for solving recurrences

Master Theorem

log 𝑎

divide a problem of size 𝑛 into 𝑎 subproblems, each of size ^𝑛

𝑏 is solved in time 𝑇 ^𝑛

𝑏 recursively

The proof is in Ch. 4.6

Should follow this format

Recursion-Tree for Master Theorem

+

𝑎 𝑎

Three Cases

•

• 𝑎 ≥ 1, the number of subproblems

• 𝑏 > 1, the factor by which the subproblem size decreases

• 𝑓(𝑛) = work to divide/combine subproblems

• Compare 𝑓 𝑛 with 𝑛^{log𝑏 𝑎}

1. Case 1: 𝑓 𝑛 grows polynomially slower than 𝑛^log𝑏𝑎 2. Case 2: 𝑓 𝑛 and 𝑛^{log𝑏 𝑎} grow at similar rates

3. Case 3: 𝑓 𝑛 grows polynomially faster than 𝑛^{log𝑏 𝑎}

Case 1:

Total cost dominated by the leaves

𝑎 𝑎

Case 1:

Total cost dominated by the leaves

𝑎 𝑎

Case 2:

Total cost evenly distributed among levels

Case 2:

Total cost evenly distributed among levels

Case 3:

Total cost dominated by root cost

𝑎 𝑎

log 𝑎

Case 3:

Total cost dominated by root cost

Master Theorem

divide a problem of size 𝑛 into 𝑎 subproblems, each of size ^𝑛

𝑏 is solved in time 𝑇 ^𝑛

𝑏 recursively

The proof is in Ch. 4.6

Examples

compare 𝑓 𝑛 with 𝑛^log𝑏𝑎

Floors and Ceilings

• Master theorem can be extended to recurrences with floors and ceilings

• The proof is in the Ch. 4.6

Theorem 1

• Case 2

Theorem 2

• Case 1

Theorem 3

• Case 2

To Be Continue…

Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw