• 沒有找到結果。

# Algorithm Design and Analysis Dynamic Programming (2)

N/A
N/A
Protected

Share "Algorithm Design and Analysis Dynamic Programming (2)"

Copied!
63
0
0

(1)

(2)

### • Homework 2 released

• Due on 11/12 (Thur) 13:00 (2.5 weeks left)

• Writing: print out the A4 hard copy and submit to NTU COOL

• Programming: submit to Online Judge – http://ada-judge.csie.ntu.edu.tw

(3)

(4)

### Outline

• Dynamic Programming

• DP #1: Rod Cutting

• DP #2: Stamp Problem

• DP #3: Sequence Alignment Problem

• Longest Common Subsequence (LCS) / Edit Distance

• Viterbi Algorithm

• Space Efficient Algorithm

• DP #4: Matrix-Chain Multiplication

• DP #5: Weighted Interval Scheduling

• DP #6: Knapsack Problem

• 0/1 Knapsack

• Unbounded Knapsack

• Multidimensional Knapsack

• Fractional Knapsack

(5)

### 動腦一下 – 囚犯問題

• 有100個死囚，隔天執行死刑，典獄長開恩給他們一個存活的機會。

• 當隔天執行死刑時，每人頭上戴一頂帽子(黑或白)排成一隊伍，在死刑執行前，由隊 伍中最後的囚犯開始，每個人可以猜測自己頭上的帽子顏色(只允許說黑或白)，猜對 則免除死刑，猜錯則執行死刑。

• 若這些囚犯可以前一天晚上先聚集討論方案，是否有好的方法可以使總共存活的囚 犯數量期望值最高？

(6)

### 猜測規則

• 囚犯排成一排，每個人可以看到前面所有人的帽子，但看不到自己及後面囚犯的。

• 由最後一個囚犯開始猜測，依序往前。

• 每個囚犯皆可聽到之前所有囚犯的猜測內容。

……

Example: 奇數者猜測內容為前面一位的帽子顏色 → 存活期望值為75人 有沒有更多人可以存活的好策略?

(7)

https://fast-poll.com/poll/9376b781

(8)

### DP#4: Matrix-Chain Multiplication

Textbook Chapter 15.2 – Matrix-chain multiplication

(9)

1

𝑛

1

2

### … 𝐴

𝑛

𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛

𝐴1.cols=𝐴2.rows

𝐴1and 𝐴2are compatible.

(10)

### • There are total 𝑝𝑟 entries

A B C

Matrix multiplication is associative: 𝐴 𝐵𝐶 = (𝐴𝐵)𝐶. The time required by

(11)

(12)

(13)

0

1

### , … , 𝑙

𝑛

• 𝑙𝑖−1 is the number of rows of matrix 𝐴𝑖

• 𝑙𝑖 is the number of columns of matrix 𝐴𝑖

1

2

### … 𝐴

𝑛

𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛

𝐴1.cols=𝐴2.rows

𝐴1and 𝐴2are compatible.

Do not need to compute the result but find the fast way to get the result!

(computing “how to fast compute” takes less time than “computing via a bad way”)

(14)

𝑛

𝑛

4𝑛

𝑛

3 2

### , or is also Ω 2

𝑛 Exercise 15.2-3

(15)

### • Subproblems

• M(i, j): the min #operations for obtaining the product of 𝐴𝑖 … 𝐴𝑗

• Goal: M(1, n)

### • Optimal substructure: suppose we know the OPT to M(i, j) , there are k cases:

• Case k: there is a cut right after Ak in OPT Matrix-Chain Multiplication Problem

Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖

Output: an order of matrix multiplications with the minimum number of operations

𝐴𝑖𝐴𝑖+1 … 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2 … 𝐴𝑗 𝑖 ≤ 𝑘 < 𝑗

(16)

### • Suppose we know the optimal solution to M(i, j) , there are k cases:

• Case k: there is a cut right after Ak in OPT

### • Recursively define the value

𝐴𝑘+1..𝑗

𝐴𝑖.rows

=𝑙𝑖−1

𝐴𝑘.cols=𝑙𝑘

𝐴𝑘+1.rows=𝑙𝑘

𝐴𝑗.cols=𝑙𝑗

𝐴𝑖𝐴𝑖+1… 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2… 𝐴𝑗 =

Matrix-Chain Multiplication Problem

Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖

Output: an order of matrix multiplications with the minimum number of operations

(17)

### • How many subproblems to solve

• #combination of the values 𝑖 and 𝑗 s.t. 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛 Matrix-Chain Multiplication Problem

Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖

Output: an order of matrix multiplications with the minimum number of operations

(18)

### Step 3: Compute Value of an OPT Solution

Matrix-Chain(n, l)

initialize two tables M[1..n][1..n] and B[1..n-1][2..n]

for i = 1 to n

M[i][i] = 0 // boundary case

for p = 2 to n // p is the chain length

for i = 1 to n – p + 1 // all i, j combinations j = i + p – 1

M[i][j] = ∞

for k = i to j – 1 // find the best k

q = M[i][k] + M[k + 1][j] + l[i - 1] * l[k] * l[j]

if q < M[i][j]

M[i][j] = q return M

(19)

### Dynamic Programming Illustration

How to decide the order of the matrix

multiplication?

1 2 3 4 5 6 n

1 0

2 0

3 0

4 0

5 0

6 0

0

n 0

(20)

### Step 4: Construct an OPT Solution by Backtracking

Matrix-Chain(n, l)

initialize two tables M[1..n][1..n] and B[1..n-1][2..n]

for i = 1 to n

M[i][i] = 0 // boundary case

for p = 2 to n // p is the chain length

for i = 1 to n – p + 1 // all i, j combinations j = i + p – 1

M[i][j] = ∞

for k = i to j – 1 // find the best k

q = M[i][k] + M[k + 1][j] + l[i - 1] * l[k] * l[j]

if q < M[i][j]

M[i][j] = q

B[i][j] = k // backtracking return M and B

Print-Optimal-Parens(B, i, j) if i == j

print 𝐴𝑖 else

print “(”

Print-Optimal-Parens(B, i, B[i][j]) Print-Optimal-Parens(B, B[i][j] + 1, j)

(21)

### Exercise

Matrix 𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒 𝑨𝟓 𝑨𝟔

Dimension 30 x 35 35 x 15 15 x 5 5 x 10 10 x 20 20 x 25

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

15,750

2,625 750

1,000

5,000 7,875

4,375

2,500

3,500 9,375

7,125

53,75 11,875

10,500 15,125

1 2 3 4 5 6

1 2 3 4 5 6

1

2

3

4

5 1

3

3

5 3

3

3 3

3 3

(22)

## DP#5: Weighted Interval Scheduling

Textbook Exercise 16.2-2

(23)

𝑖

𝑖

### • The interval scheduling problem can be solved using an “early-finish-time-first” greedy algorithm in 𝑂(𝑛) time

“Greedy Algorithm”

Next topic!

time 1

2 3 4 5 6

job index

(24)

𝑖

𝑖

𝑖

### • Output: the maximum total value obtainable from compatible jobs

1

3

3 4

3 1 1

2 3 4 5 6

job index

Assume that the requests are sorted in non-decreasing order (𝑓𝑖 ≤ 𝑓𝑗 when 𝑖 < 𝑗) 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible

e.g. 𝑝 1 = 0, 𝑝 2 = 0, 𝑝 3 = 1, 𝑝 4 = 1, 𝑝 5 = 4, 𝑝 6 = 3

(25)

### • Subproblems

• WIS(i): weighted interval scheduling for the first 𝑖 jobs

• Goal: WIS(n)

### • Optimal substructure: suppose OPT is an optimal solution to WIS(i) , there are 2 cases:

• Case 1: job 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of WIS(p(i))

• Case 2: job 𝑖 not in OPT

• OPT is an optimal solution of WIS(i-1) time

1 3

3 4

3 1 1

2 3 4 5 6

job index

2

1 3 4 5 6 7 8 9

4

1

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(26)

### • Optimal substructure: suppose OPT is an optimal solution to WIS(i) , there are 2 cases:

• Case 1: job 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of WIS(p(i))

• Case 2: job 𝑖 not in OPT

• OPT is an optimal solution of WIS(i-1)

### • Recursively define the value

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(27)

### Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

i 0 1 2 3 4 5 n

M[i]

WIS(n, s, f, v, p) M[0] = 0

for i = 1 to n

M[i] = max(v[i] + M[p[i]], M[i - 1]) return M[n]

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(28)

### Step 4: Construct an OPT Solution by Backtracking

• Bottom-up method: solve smaller subproblems first

i 0 1 2 3 4 5 6

M[i] 0 1 3 4 5 6 7

1

3

3 4

1 1

2 3 4 5

job index

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(29)

### Step 4: Construct an OPT Solution by Backtracking

WIS(n, s, f, v, p) M[0] = 0

for i = 1 to n

M[i] = max(v[i] + M[p[i]], M[i - 1]) return M[n]

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

Find-Solution(M, n) if n = 0

return {}

if v[n] + M[p[n]] > M[n-1] // case 1 return {n} ∪ Find-Solution(p[n]) return Find-Solution(n-1) // case 2

(30)

## DP#3: Knapsack (背包問題)

Textbook Exercise 16.2-2

(31)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(32)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(33)

### • Subproblems

• ZO-KP(i, w): 0-1 knapsack problem within 𝑤 capacity for the first 𝑖 items

• Goal: ZO-KP(n, W)

### • Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w) , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of ZO-KP(i - 1, w)

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

ZO-KP(i) ZO-KP(i, w)

consider the available capacity

(34)

### • Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w) , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of ZO-KP(i - 1, w)

### • Recursively define the value

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(35)

### • Bottom-up method: solve smaller subproblems first

i\w 0 1 2 3 w W

0 1 2 i n

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(36)

### • Bottom-up method: solve smaller subproblems first

i\w 0 1 2 3 4 5

0 0 0 0 0 0 0

1 0 4 4 4 4 4

2 0 4 9 13 13 13

3 0 4 9 13 20 24

i wi vi

1 1 4

2 2 9

3 4 20

𝑊 = 5

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(37)

### • Bottom-up method: solve smaller subproblems first

ZO-KP(n, v, W) for w = 0 to W

M[0, w] = 0 for i = 1 to n

for w = 0 to W if(wi > w)

M[i, w] = M[i-1, w]

else

M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(38)

### Step 4: Construct an OPT Solution by Backtracking

ZO-KP(n, v, W) for w = 0 to W

M[0, w] = 0 for i = 1 to n

for w = 0 to W if(wi > w)

M[i, w] = M[i-1, w]

else

M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]

Find-Solution(M, n, W) S = {}

w = W

for i = n to 1

if M[i, w] > M[i – 1, w] // case 1 w = w – wi

S = S ∪ {i}

(39)

### • The time complexity of 0-1 knapsack problem is Θ 𝑛𝑊

• 𝑛: number of objects

• 𝑊: knapsack’s capacity (non-negative integer)

• polynomial in the numeric value

= pseudo-polynomial in input size

= exponential in the length of the input

2

= 2𝑚 = 𝑚

(40)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(41)

### • Subproblems

• U-KP(i, w): unbounded knapsack problem with 𝑤 capacity for the first 𝑖 items

• Goal: U-KP(n, W)

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

0-1 Knapsack Problem Unbounded Knapsack Problem

each item can be chosen at most once each item can be chosen multiple times a sequence of binary choices: whether to

choose item 𝑖

a sequence of 𝑖 choices: which one (from 1 to 𝑖) to choose

Time complexity = Θ 𝑛𝑊 Time complexity = Θ 𝑛2𝑊

Can we do better?

(42)

### • Subproblems

• U-KP(w): unbounded knapsack problem with 𝑤 capacity

• Goal: U-KP(W)

U-KP(w)

### , there are 𝑛 cases:

• Case 1: item 1 in OPT

• Removing an item 1 from OPT is an optimal solution of U-KP(w – w1)

• Case 2: item 2 in OPT

• Removing an item 2 from OPT is an optimal solution of U-KP(w – w2) :

• Case 𝑛: item 𝑛 in OPT

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(43)

### • Optimal substructure: suppose OPT is an optimal solution to U-KP(w) , there are 𝑛 cases:

• Case 𝑖: item 𝑖 in OPT

• Removing an item i from OPT is an optimal solution of U-KP(w – w1)

### • Recursively define the value

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(44)

### • Bottom-up method: solve smaller subproblems first

w 0 1 2 3 4 5 W

M[w]

i wi vi

1 1 4

2 2 9

3 4 20

𝑊 = 5

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(45)

### • Bottom-up method: solve smaller subproblems first

w 0 1 2 3 4 5

M[w] 0 4 9 13 18 22

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

i wi vi

1 1 4

2 2 9

3 4 20

𝑊 = 5

(46)

### • Bottom-up method: solve smaller subproblems first

U-KP(v, W)

for w = 0 to W M[w] = 0

for w = 0 to W for i = 1 to n

if(wi <= w)

tmp = vi + M[w - wi] M[w] = max(M[w], tmp)

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(47)

### Step 4: Construct an OPT Solution by Backtracking

U-KP(v, W)

for w = 0 to W M[w] = 0

for w = 0 to W for i = 1 to n

if(wi <= w)

tmp = vi + M[w - wi] M[w] = max(M[w], tmp) return M[W]

Find-Solution(M, n, W) for i = 1 to n

C[i] = 0 // C[i] = # of item i in solution w = W

for i = i to n while w > 0

if(wi <= w && M[w] == (vi + M[w - wi])) w = w - wi

C[i] += 1 return C

(48)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(49)

### • Subproblems

• M-KP(i, w, d): multidimensional knapsack problem with 𝑤 capacity and 𝑑 size for the first 𝑖 items

• Goal: M-KP(n, W, D)

M-KP(i, w, d)

### , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of M-KP(i - 1, w, d)

Multidimensional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖

Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once

(50)

### • Optimal substructure: suppose OPT is an optimal solution to M-KP(i, w, d) , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of M-KP(i - 1, w, d)

### • Recursively define the value

Multidimensional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖

Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once

(51)

### • What is the time complexity?

Multidimensional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖

Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once

(52)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(53)

### • Input: 𝑛 items

• 𝑣𝑖,𝑗: value of 𝑗-th item in the group 𝑖

• 𝑤𝑖,𝑗: weight of 𝑗-th item in the group 𝑖

• 𝑛𝑖: number of items in group 𝑖

• 𝑛: total number of items (σ 𝑛𝑖)

• 𝐺: total number of groups

group 2 group 3

(54)

### • Subproblems

• MC-KP(w): 𝑤 capacity

• MC-KP(i, w): 𝑤 capacity for the first 𝑖 groups

• MC-KP(i, j, w): 𝑤 capacity for the first 𝑗 items from first 𝑖 groups Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

Which one is more suitable for this problem?

the constraint is for groups

(55)

### • Subproblems

• MC-KP(i, w): multi-choice knapsack problem with 𝑤 capacity for the first 𝑖 groups

• Goal: MC-KP(G, W)

𝑖

### + 1 cases:

• Case 1: no item from 𝑖-th group in OPT

• OPT is an optimal solution of MC-KP(i - 1, w)

:

• Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(56)

𝑖

### + 1 cases:

• Case 1: no item from 𝑖-th group in OPT

• OPT is an optimal solution of MC-KP(i - 1, w)

• Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT

• OPT\itemi,j is an optimal solution of MC-KP(i - 1, w – wi,j)

### • Recursively define the value

Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(57)

### • Bottom-up method: solve smaller subproblems first

i\w 0 1 2 3 w W

0 1 2 i n

Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(58)

### • Bottom-up method: solve smaller subproblems first

MC-KP(n, v, W) for w = 0 to W

M[0, w] = 0

for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w

M[i, w] = M[i - 1, w]

for j = 1 to ni // check j-th item in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])

M[i, w] = vi,j + M[i - 1, w - wi,j] return M[G, W]

Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(59)

### Step 4: Construct an OPT Solution by Backtracking

MC-KP(n, v, W) for w = 0 to W

M[0, w] = 0

for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w

M[i, w] = M[i - 1, w]

for j = 1 to ni // check items in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])

M[i, w] = vi,j + M[i - 1, w - wi,j] B[i, w] = j

return M[G, W], B[G, W]

Practice to write the pseudo code for Find-Solution()

(60)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(61)

𝑖

𝑖

𝑖

𝑖

𝑣𝑖

𝑤𝑖

### (類似CP值) first

“Greedy Algorithm”

Next topic!

Can we do better?

(62)

### • When to use DP

• Whether subproblem solutions can combine into the original solution

• When subproblems are overlapping

• Whether the problem has optimal substructure

• Common for optimization problem

### • Two ways to avoid recomputation

• Top-down with memoization

• Bottom-up method

### • Complexity analysis

• Space for tabular filling

• Size of the subproblem graph

(63)

## Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

• A concrete problem is polynomial-time solvable if there exists an algorithm that solves any concrete instance of length n in time for some constant k. • Solvable = can produce

• In the worst case, what is the growth of the function the optimal algorithm of the problem takes Design Step.

Textbook Chapter 4.3 – The substitution method for solving recurrences Textbook Chapter 4.4 – The recursion-tree method for solving recurrences Textbook Chapter 4.5 – The master

• “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution. • When to

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from

• Thinking: solve easiest case + combine smaller solutions into the original solution.. • Easy to find an

• Algorithmic design methods to solve problems efficiently (polynomial time).. • Divide

• makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.. • not always yield optimal solution; may end up at

Textbook Chapter 33.4 – Finding the closest pair of points.. Closest Pair of

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem. Prove that there is always an optimal solution to the

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem.. Prove that there is always an optimal solution to the

• Step 2: Run DFS on the transpose

Textbook Chapter 4.3 – The substitution method for solving recurrences Textbook Chapter 4.4 – The recursion-tree method for solving recurrences Textbook Chapter 4.5 – The master

Calculate the amortized cost of each operation based on the potential function 4. Calculate total amortized cost based on

[r]

[r]

[r]

[r]

creted by howard41436 edited by

[r]

 “More Joel on Software : Further Thoughts on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and Managers, and to Those

This bioinformatic machine is a PC cluster structure using special hardware to accelerate dynamic programming, genetic algorithm and data mining algorithm.. In this machine,