• 沒有找到結果。

Algorithm Design and Analysis Dynamic Programming (2)

N/A
N/A
Protected

Academic year: 2022

Share "Algorithm Design and Analysis Dynamic Programming (2)"

Copied!
63
0
0

全文

(1)

Algorithm Design and Analysis Dynamic Programming (2)

http://ada.miulab.tw

(2)

Announcement

• Homework 2 released

• Due on 11/12 (Thur) 13:00 (2.5 weeks left)

• Writing: print out the A4 hard copy and submit to NTU COOL

• Programming: submit to Online Judge – http://ada-judge.csie.ntu.edu.tw

(3)

Homework 2

(4)

Outline

• Dynamic Programming

• DP #1: Rod Cutting

• DP #2: Stamp Problem

• DP #3: Sequence Alignment Problem

• Longest Common Subsequence (LCS) / Edit Distance

• Viterbi Algorithm

• Space Efficient Algorithm

• DP #4: Matrix-Chain Multiplication

• DP #5: Weighted Interval Scheduling

• DP #6: Knapsack Problem

• 0/1 Knapsack

• Unbounded Knapsack

• Multidimensional Knapsack

• Fractional Knapsack

(5)

動腦一下 – 囚犯問題

• 有100個死囚,隔天執行死刑,典獄長開恩給他們一個存活的機會。

• 當隔天執行死刑時,每人頭上戴一頂帽子(黑或白)排成一隊伍,在死刑執行前,由隊 伍中最後的囚犯開始,每個人可以猜測自己頭上的帽子顏色(只允許說黑或白),猜對 則免除死刑,猜錯則執行死刑。

• 若這些囚犯可以前一天晚上先聚集討論方案,是否有好的方法可以使總共存活的囚 犯數量期望值最高?

(6)

猜測規則

• 囚犯排成一排,每個人可以看到前面所有人的帽子,但看不到自己及後面囚犯的。

• 由最後一個囚犯開始猜測,依序往前。

• 每個囚犯皆可聽到之前所有囚犯的猜測內容。

……

Example: 奇數者猜測內容為前面一位的帽子顏色 → 存活期望值為75人 有沒有更多人可以存活的好策略?

(7)

Vote for Your Answer

https://fast-poll.com/poll/9376b781

(8)

DP#4: Matrix-Chain Multiplication

Textbook Chapter 15.2 – Matrix-chain multiplication

(9)

Matrix-Chain Multiplication

• Input: a sequence of n matrices 𝐴

1

, … , 𝐴

𝑛

• Output: the product of 𝐴

1

𝐴

2

… 𝐴

𝑛

𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛

𝐴1.cols=𝐴2.rows

𝐴1and 𝐴2are compatible.

(10)

Observation

• Each entry takes 𝑞 multiplications

• There are total 𝑝𝑟 entries

A B C

Matrix multiplication is associative: 𝐴 𝐵𝐶 = (𝐴𝐵)𝐶. The time required by

(11)

Example

• Overall time is

= =

(12)

Example

• Overall time is

= =

(13)

Matrix-Chain Multiplication Problem

• Input: a sequence of integers 𝑙

0

, 𝑙

1

, … , 𝑙

𝑛

• 𝑙𝑖−1 is the number of rows of matrix 𝐴𝑖

• 𝑙𝑖 is the number of columns of matrix 𝐴𝑖

• Output: an order of performing 𝑛 − 1 matrix multiplications in the minimum number of operations to obtain the product of 𝐴

1

𝐴

2

… 𝐴

𝑛

𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛

𝐴1.cols=𝐴2.rows

𝐴1and 𝐴2are compatible.

Do not need to compute the result but find the fast way to get the result!

(computing “how to fast compute” takes less time than “computing via a bad way”)

(14)

Brute-Force Naïve Algorithm

• 𝑃

𝑛

: how many ways for 𝑛 matrices to be multiplied

• The solution of 𝑃

𝑛

is Catalan numbers, Ω

4𝑛

𝑛

3 2

, or is also Ω 2

𝑛 Exercise 15.2-3

(15)

Step 1: Characterize an OPT Solution

• Subproblems

• M(i, j): the min #operations for obtaining the product of 𝐴𝑖 … 𝐴𝑗

• Goal: M(1, n)

• Optimal substructure: suppose we know the OPT to M(i, j) , there are k cases:

• Case k: there is a cut right after Ak in OPT Matrix-Chain Multiplication Problem

Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖

Output: an order of matrix multiplications with the minimum number of operations

左右所花的運算量是M(i, k)M(k+1, j)的最佳解

𝐴𝑖𝐴𝑖+1 … 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2 … 𝐴𝑗 𝑖 ≤ 𝑘 < 𝑗

(16)

Step 2: Recursively Define the Value of an OPT Solution

• Suppose we know the optimal solution to M(i, j) , there are k cases:

• Case k: there is a cut right after Ak in OPT

• Recursively define the value

左右所花的運算量是M(i, k)M(k+1, j)的最佳解

𝐴𝑘+1..𝑗

𝐴𝑖.rows

=𝑙𝑖−1

𝐴𝑘.cols=𝑙𝑘

𝐴𝑘+1.rows=𝑙𝑘

𝐴𝑗.cols=𝑙𝑗

𝐴𝑖𝐴𝑖+1… 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2… 𝐴𝑗 =

Matrix-Chain Multiplication Problem

Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖

Output: an order of matrix multiplications with the minimum number of operations

(17)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

• How many subproblems to solve

• #combination of the values 𝑖 and 𝑗 s.t. 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛 Matrix-Chain Multiplication Problem

Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖

Output: an order of matrix multiplications with the minimum number of operations

(18)

Step 3: Compute Value of an OPT Solution

Matrix-Chain(n, l)

initialize two tables M[1..n][1..n] and B[1..n-1][2..n]

for i = 1 to n

M[i][i] = 0 // boundary case

for p = 2 to n // p is the chain length

for i = 1 to n – p + 1 // all i, j combinations j = i + p – 1

M[i][j] = ∞

for k = i to j – 1 // find the best k

q = M[i][k] + M[k + 1][j] + l[i - 1] * l[k] * l[j]

if q < M[i][j]

M[i][j] = q return M

(19)

Dynamic Programming Illustration

How to decide the order of the matrix

multiplication?

1 2 3 4 5 6 n

1 0

2 0

3 0

4 0

5 0

6 0

0

n 0

(20)

Step 4: Construct an OPT Solution by Backtracking

Matrix-Chain(n, l)

initialize two tables M[1..n][1..n] and B[1..n-1][2..n]

for i = 1 to n

M[i][i] = 0 // boundary case

for p = 2 to n // p is the chain length

for i = 1 to n – p + 1 // all i, j combinations j = i + p – 1

M[i][j] = ∞

for k = i to j – 1 // find the best k

q = M[i][k] + M[k + 1][j] + l[i - 1] * l[k] * l[j]

if q < M[i][j]

M[i][j] = q

B[i][j] = k // backtracking return M and B

Print-Optimal-Parens(B, i, j) if i == j

print 𝐴𝑖 else

print “(”

Print-Optimal-Parens(B, i, B[i][j]) Print-Optimal-Parens(B, B[i][j] + 1, j)

(21)

Exercise

Matrix 𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒 𝑨𝟓 𝑨𝟔

Dimension 30 x 35 35 x 15 15 x 5 5 x 10 10 x 20 20 x 25

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

15,750

2,625 750

1,000

5,000 7,875

4,375

2,500

3,500 9,375

7,125

53,75 11,875

10,500 15,125

1 2 3 4 5 6

1 2 3 4 5 6

1

2

3

4

5 1

3

3

5 3

3

3 3

3 3

(22)

DP#5: Weighted Interval Scheduling

Textbook Exercise 16.2-2

(23)

Interval Scheduling

• Input: 𝑛 job requests with start times 𝑠

𝑖

, finish times 𝑓

𝑖

• Output: the maximum number of compatible jobs

• The interval scheduling problem can be solved using an “early-finish-time- first” greedy algorithm in 𝑂(𝑛) time

“Greedy Algorithm”

Next topic!

time 1

2 3 4 5 6

job index

(24)

Weighted Interval Scheduling

• Input: 𝑛 job requests with start times 𝑠

𝑖

, finish times 𝑓

𝑖

, and values 𝑣

𝑖

• Output: the maximum total value obtainable from compatible jobs

1

3

3 4

3 1 1

2 3 4 5 6

job index

Assume that the requests are sorted in non-decreasing order (𝑓𝑖 ≤ 𝑓𝑗 when 𝑖 < 𝑗) 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible

e.g. 𝑝 1 = 0, 𝑝 2 = 0, 𝑝 3 = 1, 𝑝 4 = 1, 𝑝 5 = 4, 𝑝 6 = 3

(25)

Step 1: Characterize an OPT Solution

• Subproblems

• WIS(i): weighted interval scheduling for the first 𝑖 jobs

• Goal: WIS(n)

• Optimal substructure: suppose OPT is an optimal solution to WIS(i) , there are 2 cases:

• Case 1: job 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of WIS(p(i))

• Case 2: job 𝑖 not in OPT

• OPT is an optimal solution of WIS(i-1) time

1 3

3 4

3 1 1

2 3 4 5 6

job index

2

1 3 4 5 6 7 8 9

4

1

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(26)

Step 2: Recursively Define the Value of an OPT Solution

• Optimal substructure: suppose OPT is an optimal solution to WIS(i) , there are 2 cases:

• Case 1: job 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of WIS(p(i))

• Case 2: job 𝑖 not in OPT

• OPT is an optimal solution of WIS(i-1)

• Recursively define the value

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(27)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

i 0 1 2 3 4 5 n

M[i]

WIS(n, s, f, v, p) M[0] = 0

for i = 1 to n

M[i] = max(v[i] + M[p[i]], M[i - 1]) return M[n]

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(28)

Step 4: Construct an OPT Solution by Backtracking

• Bottom-up method: solve smaller subproblems first

i 0 1 2 3 4 5 6

M[i] 0 1 3 4 5 6 7

1

3

3 4

1 1

2 3 4 5

job index

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(29)

Step 4: Construct an OPT Solution by Backtracking

WIS(n, s, f, v, p) M[0] = 0

for i = 1 to n

M[i] = max(v[i] + M[p[i]], M[i - 1]) return M[n]

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

Find-Solution(M, n) if n = 0

return {}

if v[n] + M[p[n]] > M[n-1] // case 1 return {n} ∪ Find-Solution(p[n]) return Find-Solution(n-1) // case 2

(30)

DP#3: Knapsack (背包問題)

Textbook Exercise 16.2-2

(31)

Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊

• Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(32)

Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊

• Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(33)

Step 1: Characterize an OPT Solution

• Subproblems

• ZO-KP(i, w): 0-1 knapsack problem within 𝑤 capacity for the first 𝑖 items

• Goal: ZO-KP(n, W)

• Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w) , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of ZO-KP(i - 1, w)

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

ZO-KP(i) ZO-KP(i, w)

consider the available capacity

(34)

Step 2: Recursively Define the Value of an OPT Solution

• Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w) , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of ZO-KP(i - 1, w)

• Recursively define the value

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(35)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

i\w 0 1 2 3 w W

0 1 2 i n

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(36)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

i\w 0 1 2 3 4 5

0 0 0 0 0 0 0

1 0 4 4 4 4 4

2 0 4 9 13 13 13

3 0 4 9 13 20 24

i wi vi

1 1 4

2 2 9

3 4 20

𝑊 = 5

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(37)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

ZO-KP(n, v, W) for w = 0 to W

M[0, w] = 0 for i = 1 to n

for w = 0 to W if(wi > w)

M[i, w] = M[i-1, w]

else

M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]

0-1 Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where each item is chosen at most once

(38)

Step 4: Construct an OPT Solution by Backtracking

ZO-KP(n, v, W) for w = 0 to W

M[0, w] = 0 for i = 1 to n

for w = 0 to W if(wi > w)

M[i, w] = M[i-1, w]

else

M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]

Find-Solution(M, n, W) S = {}

w = W

for i = n to 1

if M[i, w] > M[i – 1, w] // case 1 w = w – wi

S = S ∪ {i}

(39)

Pseudo-Polynomial Time

• Polynomial: polynomial in the length of the input (#bits for the input)

• Pseudo-polynomial: polynomial in the numeric value

• The time complexity of 0-1 knapsack problem is Θ 𝑛𝑊

• 𝑛: number of objects

• 𝑊: knapsack’s capacity (non-negative integer)

• polynomial in the numeric value

= pseudo-polynomial in input size

= exponential in the length of the input

• Note: the size of the representation of 𝑊 is log

2

𝑊

= 2𝑚 = 𝑚

(40)

Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊

• Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(41)

Step 1: Characterize an OPT Solution

• Subproblems

• U-KP(i, w): unbounded knapsack problem with 𝑤 capacity for the first 𝑖 items

• Goal: U-KP(n, W)

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

0-1 Knapsack Problem Unbounded Knapsack Problem

each item can be chosen at most once each item can be chosen multiple times a sequence of binary choices: whether to

choose item 𝑖

a sequence of 𝑖 choices: which one (from 1 to 𝑖) to choose

Time complexity = Θ 𝑛𝑊 Time complexity = Θ 𝑛2𝑊

Can we do better?

(42)

Step 1: Characterize an OPT Solution

• Subproblems

• U-KP(w): unbounded knapsack problem with 𝑤 capacity

• Goal: U-KP(W)

• Optimal substructure: suppose OPT is an optimal solution to

U-KP(w)

, there are 𝑛 cases:

• Case 1: item 1 in OPT

• Removing an item 1 from OPT is an optimal solution of U-KP(w – w1)

• Case 2: item 2 in OPT

• Removing an item 2 from OPT is an optimal solution of U-KP(w – w2) :

• Case 𝑛: item 𝑛 in OPT

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(43)

Step 2: Recursively Define the Value of an OPT Solution

• Optimal substructure: suppose OPT is an optimal solution to U-KP(w) , there are 𝑛 cases:

• Case 𝑖: item 𝑖 in OPT

• Removing an item i from OPT is an optimal solution of U-KP(w – w1)

• Recursively define the value

只考慮背包還裝的下的情形

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(44)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

w 0 1 2 3 4 5 W

M[w]

i wi vi

1 1 4

2 2 9

3 4 20

𝑊 = 5

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(45)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

w 0 1 2 3 4 5

M[w] 0 4 9 13 18 22

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

i wi vi

1 1 4

2 2 9

3 4 20

𝑊 = 5

(46)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

U-KP(v, W)

for w = 0 to W M[w] = 0

for w = 0 to W for i = 1 to n

if(wi <= w)

tmp = vi + M[w - wi] M[w] = max(M[w], tmp)

Unbounded Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity

(47)

Step 4: Construct an OPT Solution by Backtracking

U-KP(v, W)

for w = 0 to W M[w] = 0

for w = 0 to W for i = 1 to n

if(wi <= w)

tmp = vi + M[w - wi] M[w] = max(M[w], tmp) return M[W]

Find-Solution(M, n, W) for i = 1 to n

C[i] = 0 // C[i] = # of item i in solution w = W

for i = i to n while w > 0

if(wi <= w && M[w] == (vi + M[w - wi])) w = w - wi

C[i] += 1 return C

(48)

Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊

• Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(49)

Step 1: Characterize an OPT Solution

• Subproblems

• M-KP(i, w, d): multidimensional knapsack problem with 𝑤 capacity and 𝑑 size for the first 𝑖 items

• Goal: M-KP(n, W, D)

• Optimal substructure: suppose OPT is an optimal solution to

M-KP(i, w, d)

, there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of M-KP(i - 1, w, d)

Multidimensional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖

Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once

(50)

Step 2: Recursively Define the Value of an OPT Solution

• Optimal substructure: suppose OPT is an optimal solution to M-KP(i, w, d) , there are 2 cases:

• Case 1: item 𝑖 in OPT

• OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of M-KP(i - 1, w, d)

• Recursively define the value

Multidimensional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖

Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once

(51)

Exercise

• Step 3: Compute Value of an OPT Solution

• Step 4: Construct an OPT Solution by Backtracking

• What is the time complexity?

Multidimensional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖

Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once

(52)

Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊

• Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(53)

Multiple-Choice Knapsack Problem

• Input: 𝑛 items

• 𝑣𝑖,𝑗: value of 𝑗-th item in the group 𝑖

• 𝑤𝑖,𝑗: weight of 𝑗-th item in the group 𝑖

• 𝑛𝑖: number of items in group 𝑖

• 𝑛: total number of items (σ 𝑛𝑖)

• 𝐺: total number of groups

• Output: the maximum value for the knapsack with capacity of 𝑊, where the item from each group can be selected at most once

group 2 group 3

(54)

Step 1: Characterize an OPT Solution

• Subproblems

• MC-KP(w): 𝑤 capacity

• MC-KP(i, w): 𝑤 capacity for the first 𝑖 groups

• MC-KP(i, j, w): 𝑤 capacity for the first 𝑗 items from first 𝑖 groups Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

Which one is more suitable for this problem?

the constraint is for groups

(55)

Step 1: Characterize an OPT Solution

• Subproblems

• MC-KP(i, w): multi-choice knapsack problem with 𝑤 capacity for the first 𝑖 groups

• Goal: MC-KP(G, W)

• Optimal substructure: suppose OPT is an optimal solution to MC-KP(i, w) , for the group 𝑖, there are 𝑛

𝑖

+ 1 cases:

• Case 1: no item from 𝑖-th group in OPT

• OPT is an optimal solution of MC-KP(i - 1, w)

:

• Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(56)

Step 2: Recursively Define the Value of an OPT Solution

• Optimal substructure: suppose OPT is an optimal solution to MC-KP(i, w) , for the group 𝑖, there are 𝑛

𝑖

+ 1 cases:

• Case 1: no item from 𝑖-th group in OPT

• OPT is an optimal solution of MC-KP(i - 1, w)

• Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT

• OPT\itemi,j is an optimal solution of MC-KP(i - 1, w – wi,j)

• Recursively define the value

Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(57)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

i\w 0 1 2 3 w W

0 1 2 i n

Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(58)

Step 3: Compute Value of an OPT Solution

• Bottom-up method: solve smaller subproblems first

MC-KP(n, v, W) for w = 0 to W

M[0, w] = 0

for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w

M[i, w] = M[i - 1, w]

for j = 1 to ni // check j-th item in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])

M[i, w] = vi,j + M[i - 1, w - wi,j] return M[G, W]

Multiple-Choice Knapsack Problem

Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)

Output: the max value within 𝑊 capacity, where each group is chosen at most once

(59)

Step 4: Construct an OPT Solution by Backtracking

MC-KP(n, v, W) for w = 0 to W

M[0, w] = 0

for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w

M[i, w] = M[i - 1, w]

for j = 1 to ni // check items in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])

M[i, w] = vi,j + M[i - 1, w - wi,j] B[i, w] = j

return M[G, W], B[G, W]

Practice to write the pseudo code for Find-Solution()

(60)

Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊

• Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(61)

Fractional Knapsack Problem

• Input: 𝑛 items where 𝑖-th item has value 𝑣

𝑖

and weighs 𝑤

𝑖

(𝑣

𝑖

and 𝑤

𝑖

are positive integers)

• Output: the maximum value for the knapsack with capacity of 𝑊, where we can take any fraction of items

• Dynamic programming algorithm should work

• Choose maximal

𝑣𝑖

𝑤𝑖

(類似CP值) first

“Greedy Algorithm”

Next topic!

Can we do better?

(62)

Concluding Remarks

• “Dynamic Programming”: solve many subproblems in polynomial time for which a naïve approach would take exponential time

• When to use DP

• Whether subproblem solutions can combine into the original solution

• When subproblems are overlapping

• Whether the problem has optimal substructure

• Common for optimization problem

• Two ways to avoid recomputation

• Top-down with memoization

• Bottom-up method

• Complexity analysis

• Space for tabular filling

• Size of the subproblem graph

(63)

Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw

參考文獻

相關文件

• A concrete problem is polynomial-time solvable if there exists an algorithm that solves any concrete instance of length n in time for some constant k. • Solvable = can produce

• In the worst case, what is the growth of the function the optimal algorithm of the problem takes Design Step.

Textbook Chapter 4.3 – The substitution method for solving recurrences Textbook Chapter 4.4 – The recursion-tree method for solving recurrences Textbook Chapter 4.5 – The master

• “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution. • When to

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from

• Thinking: solve easiest case + combine smaller solutions into the original solution.. • Easy to find an

• Algorithmic design methods to solve problems efficiently (polynomial time).. • Divide

• makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.. • not always yield optimal solution; may end up at

Textbook Chapter 33.4 – Finding the closest pair of points.. Closest Pair of

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem. Prove that there is always an optimal solution to the

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem.. Prove that there is always an optimal solution to the

• Step 2: Run DFS on the transpose

Textbook Chapter 4.3 – The substitution method for solving recurrences Textbook Chapter 4.4 – The recursion-tree method for solving recurrences Textbook Chapter 4.5 – The master

Calculate the amortized cost of each operation based on the potential function 4. Calculate total amortized cost based on

[r]

[r]

[r]

[r]

依賴背包問題 and

creted by howard41436 edited by

[r]

 “More Joel on Software : Further Thoughts on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and Managers, and to Those

This bioinformatic machine is a PC cluster structure using special hardware to accelerate dynamic programming, genetic algorithm and data mining algorithm.. In this machine,