**Algorithm Design and Analysis** **Divide and Conquer (2)**

**http://ada.miulab.tw**

**Outline**

• Recurrence (遞迴)

• Divide-and-Conquer

• D&C #1: Tower of Hanoi (河內塔)

• D&C #2: Merge Sort

• D&C #3: Bitonic Champion

• D&C #4: Maximum Subarray

• Solving Recurrences

### • Substitution Method

### • Recursion-Tree Method

### • Master Method

• D&C #5: Matrix Multiplication

• D&C #6: Selection Problem

• D&C #7: Closest Pair of Points Problem

Divide-and-Conquer 之神乎奇技

Divide-and-Conquer 首部曲

**What is Divide-and-Conquer?**

### • Solve a problem recursively

### • Apply three steps at each level of the recursion

**1. Divide the problem into a number of subproblems that are smaller instances of the **

same problem (比較小的同樣問題)
**2. Conquer the subproblems by solving them recursively**

*If the subproblem sizes are small enough*

• then solve the subproblems

• else recursively solve itself

**3. Combine**

the solutions to the subproblems into the solution for the original problem
base case recursive case

**Solving Recurrences**

Textbook Chapter 4.3 – The substitution method for solving recurrences Textbook Chapter 4.4 – The recursion-tree method for solving recurrences Textbook Chapter 4.5 – The master method for solving recurrences

**D&C Algorithm Time Complexity**

### • 𝑇 𝑛 : running time for input size 𝑛

### • 𝐷 𝑛 : time of Divide for input size 𝑛

### • 𝐶 𝑛 : time of Combine for input size 𝑛

### • 𝑎: number of subproblems

### • 𝑛/𝑏: size of each subproblem

**Solving Recurrences**

**1. Substitution Method (取代法)**

• Guess a bound and then prove by induction

**2. Recursion-Tree Method (遞迴樹法)**

• Expand the recurrence into a tree and sum up the cost

**3. Master Method (套公式大法/大師法)**

• Apply Master Theorem to a specific form of recurrences

### • Useful simplification tricks

• Ignore floors, ceilings, boundary conditions (proof in Ch. 4.6)

*• Assume base cases are constant (for small n) *

**Substitution Method**

Textbook Chapter 4.3 – The substitution method for solving recurrences

**Review**

### • Time Complexity for Merge Sort

### • Theorem

### • Proof

• There exists positive constant 𝑎, 𝑏 s.t.

• Use induction to prove

• n = 1, trivial

• n > 1,

**Substitution Method (取代法)**

guess a bound and then prove by induction

**Substitution Method (取代法)**

• Guess the form of the solution

• Verify by mathematical induction (數學歸納法)

• Prove it works for 𝑛 = 1

• Prove that if it works for 𝑛 = 𝑚, then it works for 𝑛 = 𝑚 + 1

→ It can work for all positive integer 𝑛

• Solve constants to show that the solution works

• Prove 𝑂 and Ω separately

### 1. Guess

### 2. Verify

### 3. Solve

**Substitution Method Example**

### • Proof

### •

### There exists positive constants 𝑛

_{0}

### , 𝑐 s.t. for all 𝑛 ≥ 𝑛

_{0}

### ,

### • Use induction to find the constants 𝑛

_{0}

### , 𝑐

• n = 1, trivial

• n > 1,

### • holds when

e.g.

Inductive hypothesis

Guess

Verify

Solve

**Substitution Method Example**

### • Proof

### •

### There exists positive constants 𝑛

_{0}

### , 𝑐 s.t. for all 𝑛 ≥ 𝑛

_{0}

### ,

### • Use induction to find the constants 𝑛

_{0}

### , 𝑐

• n = 1, trivial

• n > 1,

Inductive hypothesis

Tighter upper bound?

証不出來…

猜錯了？還是推導錯了？

沒猜錯 推導也沒錯 這是取代法的小盲點

**Substitution Method Example**

### • Proof

### •

### There exists positive constants 𝑛

_{0}

### , 𝑐

_{1}

### , 𝑐

_{2}

### s.t. for all 𝑛 ≥ 𝑛

_{0}

### ,

### • Use induction to find the constants 𝑛

_{0}

### ,𝑐

_{1}

### , 𝑐

_{2}

• n = 1, holds for

• n > 1,

### • holds when

e.g.

Inductive hypothesis

Guess

Verify

Solve Strengthen the inductive hypothesis

by subtracting a low-order term

**Useful Tricks**

• Guess based on seen recurrences

• Use the recursion-tree method

• From loose bound to tight bound

• Strengthen the inductive hypothesis by subtracting a low-order term

• Change variables

• E.g.,

1. Change variable:

2. Change variable again:

3. Solve recurrence

**Recursion-Tree Method**

Textbook Chapter 4.4 – The recursion-tree method for solving recurrences

**Review**

### • Time Complexity for Merge Sort

### • Theorem

### • Proof

2^{nd} expansion
1^{st} expansion

k^{th} expansion

The expansion stops when 2^{𝑘} = 𝑛

**Recursion-Tree Method (遞迴樹法)**

Expand the recurrence into a tree and sum up the cost

**Recursion-Tree Method (遞迴樹法)**

### • Expand a recurrence into a tree

### • Sum up the cost of all nodes as a good guess

### • Verify the guess as in the substitution method

### • Advantages

• Promote intuition

• Generate good guesses for the substitution method

1. Expand

2. Sumup

3. Verify

**Recursion-Tree Example**

**Recursion-Tree Example**

**Recursion-Tree Example**

**Recursion-Tree Example**

**+**

**Master Theorem**

Textbook Chapter 4.4 – The recursion-tree method for solving recurrences

**Master Theorem**

log 𝑎

divide a problem of size 𝑛 into 𝑎 subproblems, each of size ^{𝑛}

𝑏 is solved in time 𝑇 ^{𝑛}

𝑏 recursively

The proof is in Ch. 4.6

Should follow this format

**Recursion-Tree for Master Theorem**

**+**

𝑎 𝑎

**Three Cases**

•

• 𝑎 ≥ 1, the number of subproblems

• 𝑏 > 1, the factor by which the subproblem size decreases

• 𝑓(𝑛) = work to divide/combine subproblems

• Compare 𝑓 𝑛 with 𝑛^{log}^{𝑏} ^{𝑎}

1. Case 1: 𝑓 𝑛 grows polynomially slower than 𝑛^{log}^{𝑏}^{𝑎}
2. Case 2: 𝑓 𝑛 and 𝑛^{log}^{𝑏} ^{𝑎} grow at similar rates

3. Case 3: 𝑓 𝑛 grows polynomially faster than 𝑛^{log}^{𝑏} ^{𝑎}

**Case 1: **

**Total cost dominated by the leaves**

𝑎 𝑎

**Case 1: **

**Total cost dominated by the leaves**

𝑎 𝑎

**Case 2: **

**Total cost evenly distributed among levels**

**Case 2: **

**Total cost evenly distributed among levels**

**Case 3: **

**Total cost dominated by root cost**

𝑎 𝑎

log 𝑎

**Case 3: **

**Total cost dominated by root cost**

**Master Theorem**

divide a problem of size 𝑛 into 𝑎 subproblems, each of size ^{𝑛}

𝑏 is solved in time 𝑇 ^{𝑛}

𝑏 recursively

The proof is in Ch. 4.6

**Examples**

compare 𝑓 𝑛 with 𝑛^{log}^{𝑏}^{𝑎}

**Floors and Ceilings**

• Master theorem can be extended to recurrences with floors and ceilings

• The proof is in the Ch. 4.6

**Theorem 1**

### • Case 2

**Theorem 2**

### • Case 1

**Theorem 3**

### • Case 2

**To Be Continue…**

## Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw