# Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

## Full text

(1)

Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

(2)

### Mini-HW 3 released

Due on 10/12 (Thu) 17:20

Print out the A4 hard copy and submit before the lecture finishes

### Homework 1 released

Due on 10/19 (Thur) 17:20 (2 weeks left)

Writing: print out the A4 hard copy and submit before the lecture finishes

Programming: submit to Online Judge – http://ada-judge.csie.org

### Mid-term date changed

Original: 11/09 (Thu)

New: 11/16 (Thu)

(3)

3

(4)

Recurrence (遞迴)

Divide-and-Conquer

D&C #1: Tower of Hanoi (河內塔)

D&C #2: Merge Sort

D&C #3: Bitonic Champion

D&C #4: Maximum Subarray

Solving Recurrences

Substitution Method

Recursion-Tree Method

Master Method

Divide-and-Conquer 首部曲

(5)

1.

2.

### Conquer the subproblems by solving them recursivelyIf the subproblem sizes are small enough

then solve the subproblems

else recursively solve itself

3.

### Combine the solutions to the subproblems into the solution for the original problem

5

base case recursive case

(6)

Textbook Chapter 4.3 – The substitution method for solving recurrences

## 6

(7)

7

(8)

###  Useful simplification tricks

(9)

Textbook Chapter 4.3 – The substitution method for solving recurrences

## 9

(10)

### Proof

There exists positive constant 𝑎, 𝑏 s.t.

Use induction to prove

n = 1, trivial

n > 1, Substitution Method (取代法)

(11)

11

1. Guess

2. Verify

3. Solve

(12)

### Proof

There exists positive constants 𝑛0, 𝑐 s.t. for all 𝑛 ≥ 𝑛0,

Use induction to find the constants 𝑛0, 𝑐

n = 1, trivial

n > 1,

Inductive hypothesis

Guess Verify

(13)

### Proof

There exists positive constants 𝑛0, 𝑐 s.t. for all 𝑛 ≥ 𝑛0,

Use induction to find the constants 𝑛0, 𝑐

n = 1, trivial

n > 1,

13

Inductive hypothesis

Tighter upper bound?

(14)

### Proof

There exists positive constants 𝑛0, 𝑐1, 𝑐2 s.t. for all 𝑛 ≥ 𝑛0,

Use induction to find the constants 𝑛0,𝑐1, 𝑐2

n = 1, holds for

n > 1,

Inductive hypothesis

Guess

Verify

Strengthen the inductive hypothesis by subtracting a low-order term

(15)

1.

2.

3.

### Solve recurrence

15

(16)

Textbook Chapter 4.4 – The recursion-tree method for solving recurrences

## 16

(17)

### Proof

17

2nd expansion 1st expansion

kth expansion

The expansion stops when 2𝑘 = 𝑛

Recursion-Tree Method (遞迴樹法)

Expand the recurrence into a tree and sum up the cost

(18)

1. Expand

2. Sumup

3. Verify

(19)

19

(20)
(21)

21

(22)

### +

(23)

Textbook Chapter 4.5 – The master method for solving recurrences

## 23

(24)

divide a problem of size 𝑛 into 𝑎 subproblems each of size 𝑛𝑏 is solved in time 𝑇 𝑛𝑏 recursively

The proof is in Ch. 4.6

(25)

25

𝑎 𝑎

(26)

log𝑏 𝑎

1.

log𝑏𝑎

2.

log𝑏 𝑎

### grow at similar rates

(27)

27

𝑎 𝑎

𝑓 𝑛 grows polynomially slower than 𝑛log𝑏 𝑎

(28)
(29)

29

𝑎 𝑎

𝑓 𝑛 and 𝑛log𝑏 𝑎 grow at similar rates

(30)
(31)

31

𝑎 𝑎

𝑓 𝑛 grows polynomially faster than 𝑛log𝑏 𝑎

(32)
(33)

compare 𝑓 𝑛 with 𝑛log𝑏𝑎 33

divide a problem of size 𝑛 into 𝑎 subproblems each of size 𝑛𝑏 is solved in time 𝑇 𝑛𝑏 recursively

The proof is in Ch. 4.6

(34)

compare 𝑓 𝑛 with 𝑛log𝑏 𝑎

(35)

35

(36)

(37)

37

(38)

### Case 2

(39)

Textbook Chapter 4.2 – Strassen’s algorithm for matrix multiplication

## 39

(40)
(41)

41

2

A B C

(42)
(43)

𝑘

### for simplicity

Otherwise, we can increase 𝑛 s.t. 𝑛 = 2 log2𝑛

𝑛 may not be twice large as the original in this modification

43

A11 A12

A21 A22

B11 B12

B21 B22

C11 C12

C21 C22

(44)

Combine

Conquer Divide

MatrixMultiply(n, A, B) //base case

if n == 1 __ _return AB

//recursive case

Divide A and B into n/2 by n/2 submatrices

C11 = MatrixMultiply(n/2,A11,B11) + MatrixMultiply(n/2,A12,B21) C21 = MatrixMultiply(n/2,A11,B12) + MatrixMultiply(n/2,A12,B22) C21 = MatrixMultiply(n/2,A21,B11) + MatrixMultiply(n/2,A22,B21) C22 = MatrixMultiply(n/2,A21,B12) + MatrixMultiply(n/2,A22,B22) return C

(45)

3

𝑙𝑜𝑔27

2.807

### The key idea is to reduce the number of recursive calls

From 8 recursive calls to 7 recursive calls

At the cost of extra addition and subtraction operations

45

(46)

### 𝐶 = 𝐴 × 𝐵

2 + 1×

1 + 1×

1 − 1×

1 − 1×

1 + 1×

2 + 1 − 1 + 1 + 2 + 1 −

(47)

### Practice

47

(48)

Conquer

Divide

Strassen(n, A, B) // base case if n == 1 ___ return AB

// recursive case

Divide A and B into n/2 by n/2 submatrices M1 = Strassen(n/2, A11+A22, B11+B22)

M2 = Strassen(n/2, A21+A22, B11) M3 = Strassen(n/2, A11, B12-B22) M4 = Strassen(n/2, A22, B21-B11) M5 = Strassen(n/2, A11+A12, B22) M6 = Strassen(n/2, A11-A21, B11+B12) M7 = Strassen(n/2, A12-A22, B21+B22) C = M + M - M + M

𝑇 𝑛 = time for running Strassen(n,A,B)

(49)

1.

2.

### Less numerical stable than the naïve approach

Larger errors accumulate in non-integer computation due to limited precision 3.

4.

49

(50)

### Each algorithm gives an upper bound

(51)

51

(52)

Textbook Chapter 9.3 – Selection in worst-case linear time

## 52

(53)

53

(54)

3 7 9 17 5 2 21 18 33 4

(55)

### If the sorting problem can be solved in 𝑂 𝑓 𝑛 , so can the selection problem based on the algorithm design

Step 1: sort A into increasing order

Step 2: output 𝐴[𝑛 − 𝑘 + 1]

55

(56)

Can we make the upper bound better if we do not sort them?

(57)

57

### Upper bounds in terms of #comparisons

3n + o(n) by Schonhage, Paterson, and Pippenger (JCSS 1975).

2.95n by Dor and Zwick (SODA 1995, SIAM Journal on Computing 1999).

### Lower bounds in terms of #comparisons

2n+o(n) by Bent and John (STOC 1985)

(2+2-80)n by Dor and Zwick (FOCS 1996, SIAM Journal on Discrete Math 2001).

(58)

### Idea

Select a pivot and divide the inputs into two subproblems

If 𝑘 ≤ 𝑋> , we find the 𝑘-th largest

If 𝑘 > 𝑋> , we find the 𝑘 − 𝑋> -th largest pivot

a

(59)

59

(60)
(61)

61

small number  large number

(62)

Larger than MoM Smaller than MoM

(63)

### Three cases

1. If 𝑘 ≤ 𝑋> , then output the 𝑘-th largest number in 𝑋>

2. If 𝑘 = 𝑋> + 1, then output MoM

3. If 𝑘 > 𝑋> + 1, then output the 𝑘 − 𝑋> − 1 -th largest number in 𝑋<

### Practice to prove by induction

63

Smaller than MoM Larger than MoM MoM

(64)

<

### or X

>

(65)

65

Selection(X, k) // base case if |X| <= 4

__ sort X and return X[k]

// recursive case

Divide X into |X|/5 groups with size 5 M[i] = median from group i

MoM = Selection(M, |M|/2) for i = 1 … |X|

if X[i] > MoM

insert X[i] into X2 else

insert X[i] into X1 if |X2| == k – 1

return x

if |X2| > k – 1

return Selection(X2, k)

return Selection(X1, k - |X2| - 1)

(66)

delete delete

(67)

𝑇 𝑛 = time for running Selection(X, k) with |X| = n

Intuition

67

(68)

### Proof

There exists positive constant 𝑎, 𝑏 s.t.

Use induction to prove

n = 1, 𝑎 > 𝑐

n > 1,

Inductive

(69)

69

(70)

Textbook Chapter 33.4 – Finding the closest pair of points

## 70

(71)

𝑖

𝑖

𝑖

𝑖

𝑗

### that are closest

“Closest”: smallest Euclidean distance

Euclidean distance between 𝑝𝑖 and 𝑝𝑗:

71

### Brute-force algorithm

Check all pairs of points:

Θ 𝐶2𝑛 = Θ 𝑛2

(72)

### 1D:

Sort all points

Scan the sorted points to find the closest pair in one pass

We only need to examine the adjacent points

(73)

### Combine: find closet pair with one point in each region, and return the best of three solutions

73

left-min = 10

right-min = 13 cross-min = 7

(74)

### Algo 2: only consider points within 𝛿 of the cut, 𝛿 = min{l−min, r−min}

Other pairs of points must have distance larger than 𝛿

left-min = 10

right-min = 13 cross-min = 7

𝛿 𝛿

(75)

### Algo 3: only consider pairs within 𝛿 × 2𝛿 blocks

Obs 1: every pair with smaller than 𝛿 distance must appear in a 𝛿 × 2𝛿 block

75

2𝛿區塊內怎麼辦

(76)

### Algo 3: only consider pairs within 𝛿 × 2𝛿 blocks

Obs 1: every pair with smaller than 𝛿 distance must appear in a 𝛿 × 2𝛿 block

Obs 2: there are at most 8 points in a 𝛿 × 2𝛿 block

Each 𝛿/2 × 𝛿/2 block contains at most 1 point, otherwise the distance returned from left/right region should be smaller than 𝛿

(77)

### Algo 3: only consider pairs within 𝛿 × 2𝛿 blocks

Obs 1: every pair with smaller than 𝛿 distance must appear in a 𝛿 × 2𝛿 block

Obs 2: there are at most 8 points in a 𝛿 × 2𝛿 block

pi 77

pi+4

pi+2 pi+5

pi+3

Find-closet-pair-across-regions 1. Sort the points by y-values within 𝛿 of the

cut (yellow region)

2. For the sorted point 𝑝𝑖, compute the distance with 𝑝𝑖+1, 𝑝𝑖+2, …, 𝑝𝑖+7

3. Return the smallest one

### At most 7 distance calculations needed

(78)

Closest-Pair(P)

// termination condition (base case)

if |P| <= 3 brute-force finding closest pair and return it // Divide

find a vertical line L s.t. both planes_contain half of the points // Conquer (by recursion)

left-pair, left-min = Closest-Pair(points in the left) right-pair, right-min = Closest-Pair(points in the right) // Combine

delta = min{left-min, right-min}

remove points that are delta or more away from L // Obs 1 sort remaining points by y-coordinate into p0, …, pk

for point pi:

____compute distances with pi+1, pi+2, …, pi+7_// Obs 2 ____update delta if a closer pair is found

return the closest pair and its distance

(79)

### Idea: do not sort inside the recursive case

79

Closest-Pair(P)

sort P by x- and y-coordinate and store in Px and Py // termination condition (base case)

if |P| <= 3 brute-force finding closest pair and return it // Divide

find a vertical line L s.t. both planes_contain half of the points // Conquer (by recursion)

left-pair, left-min = Closest-Pair(points in the left) right-pair, right-min = Closest-Pair(points in the right) // Combine

delta = min{left-min, right-min}

remove points that are delta or more away from L // Obs 1 for point pi in sorted candidates

____compute distances with pi+1, pi+2, …, pi+7_// Obs 2 ____update delta if a closer pair is found

return the closest pair and its distance

(80)

###  𝑂(𝑛) algorithm

Chapter 13.7 of Algorithm Design by Kleinberg & Tardos

Samir Khuller and Yossi Matias. 1995. A simple randomized sieve

algorithm for the closest-pair problem. Inf. Comput. 118, 1 (April 1995), 34-37.

(81)

### Example.

D&C algo for Fibonacci:

Bottom-up algo for Fibonacci:

81

1. Divide

2. Conquer

3. Combine

Fibonacci(n) if n < 2 ____return 1

a=1 a=1

for i = 2 … n

____a[i]=a[i-1]+a[i-2]

return a[n]

Our next topic: Dynamic Programming

“a technique for solving problems with overlapping subproblems”

(82)

## 82

Important announcement will be sent to @ntu.edu.tw mailbox

& post to the course website

Updating...

## References

Related subjects :