Algorithm Design and Analysis Dynamic Programming (2)
http://ada.miulab.tw
Announcement
• Homework 2 released
• Due on 11/12 (Thur) 13:00 (2.5 weeks left)
• Writing: print out the A4 hard copy and submit to NTU COOL
• Programming: submit to Online Judge – http://ada-judge.csie.ntu.edu.tw
Homework 2
Outline
• Dynamic Programming
• DP #1: Rod Cutting
• DP #2: Stamp Problem
• DP #3: Sequence Alignment Problem
• Longest Common Subsequence (LCS) / Edit Distance
• Viterbi Algorithm
• Space Efficient Algorithm
• DP #4: Matrix-Chain Multiplication
• DP #5: Weighted Interval Scheduling
• DP #6: Knapsack Problem
• 0/1 Knapsack
• Unbounded Knapsack
• Multidimensional Knapsack
• Fractional Knapsack
動腦一下 – 囚犯問題
• 有100個死囚,隔天執行死刑,典獄長開恩給他們一個存活的機會。
• 當隔天執行死刑時,每人頭上戴一頂帽子(黑或白)排成一隊伍,在死刑執行前,由隊 伍中最後的囚犯開始,每個人可以猜測自己頭上的帽子顏色(只允許說黑或白),猜對 則免除死刑,猜錯則執行死刑。
• 若這些囚犯可以前一天晚上先聚集討論方案,是否有好的方法可以使總共存活的囚 犯數量期望值最高?
猜測規則
• 囚犯排成一排,每個人可以看到前面所有人的帽子,但看不到自己及後面囚犯的。
• 由最後一個囚犯開始猜測,依序往前。
• 每個囚犯皆可聽到之前所有囚犯的猜測內容。
……
Example: 奇數者猜測內容為前面一位的帽子顏色 → 存活期望值為75人 有沒有更多人可以存活的好策略?
Vote for Your Answer
https://fast-poll.com/poll/9376b781
DP#4: Matrix-Chain Multiplication
Textbook Chapter 15.2 – Matrix-chain multiplication
Matrix-Chain Multiplication
• Input: a sequence of n matrices 𝐴
1, … , 𝐴
𝑛• Output: the product of 𝐴
1𝐴
2… 𝐴
𝑛𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛
𝐴1.cols=𝐴2.rows
𝐴1and 𝐴2are compatible.
Observation
• Each entry takes 𝑞 multiplications
• There are total 𝑝𝑟 entries
A B C
Matrix multiplication is associative: 𝐴 𝐵𝐶 = (𝐴𝐵)𝐶. The time required by
Example
• Overall time is
= =
Example
• Overall time is
= =
Matrix-Chain Multiplication Problem
• Input: a sequence of integers 𝑙
0, 𝑙
1, … , 𝑙
𝑛• 𝑙𝑖−1 is the number of rows of matrix 𝐴𝑖
• 𝑙𝑖 is the number of columns of matrix 𝐴𝑖
• Output: an order of performing 𝑛 − 1 matrix multiplications in the minimum number of operations to obtain the product of 𝐴
1𝐴
2… 𝐴
𝑛𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛
𝐴1.cols=𝐴2.rows
𝐴1and 𝐴2are compatible.
Do not need to compute the result but find the fast way to get the result!
(computing “how to fast compute” takes less time than “computing via a bad way”)
Brute-Force Naïve Algorithm
• 𝑃
𝑛: how many ways for 𝑛 matrices to be multiplied
• The solution of 𝑃
𝑛is Catalan numbers, Ω
4𝑛𝑛
3 2
, or is also Ω 2
𝑛 Exercise 15.2-3Step 1: Characterize an OPT Solution
• Subproblems
• M(i, j): the min #operations for obtaining the product of 𝐴𝑖 … 𝐴𝑗
• Goal: M(1, n)
• Optimal substructure: suppose we know the OPT to M(i, j) , there are k cases:
• Case k: there is a cut right after Ak in OPT Matrix-Chain Multiplication Problem
Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖
Output: an order of matrix multiplications with the minimum number of operations
左右所花的運算量是M(i, k)及M(k+1, j)的最佳解
𝐴𝑖𝐴𝑖+1 … 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2 … 𝐴𝑗 𝑖 ≤ 𝑘 < 𝑗
Step 2: Recursively Define the Value of an OPT Solution
• Suppose we know the optimal solution to M(i, j) , there are k cases:
• Case k: there is a cut right after Ak in OPT
• Recursively define the value
左右所花的運算量是M(i, k)及M(k+1, j)的最佳解
𝐴𝑘+1..𝑗
𝐴𝑖.rows
=𝑙𝑖−1
𝐴𝑘.cols=𝑙𝑘
𝐴𝑘+1.rows=𝑙𝑘
𝐴𝑗.cols=𝑙𝑗
𝐴𝑖𝐴𝑖+1… 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2… 𝐴𝑗 =
Matrix-Chain Multiplication Problem
Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖
Output: an order of matrix multiplications with the minimum number of operations
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
• How many subproblems to solve
• #combination of the values 𝑖 and 𝑗 s.t. 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛 Matrix-Chain Multiplication Problem
Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛 indicating the dimensionality of 𝐴𝑖
Output: an order of matrix multiplications with the minimum number of operations
Step 3: Compute Value of an OPT Solution
Matrix-Chain(n, l)
initialize two tables M[1..n][1..n] and B[1..n-1][2..n]
for i = 1 to n
M[i][i] = 0 // boundary case
for p = 2 to n // p is the chain length
for i = 1 to n – p + 1 // all i, j combinations j = i + p – 1
M[i][j] = ∞
for k = i to j – 1 // find the best k
q = M[i][k] + M[k + 1][j] + l[i - 1] * l[k] * l[j]
if q < M[i][j]
M[i][j] = q return M
Dynamic Programming Illustration
How to decide the order of the matrix
multiplication?
1 2 3 4 5 6 … n
1 0
2 0
3 0
4 0
5 0
6 0
: 0
n 0
Step 4: Construct an OPT Solution by Backtracking
Matrix-Chain(n, l)
initialize two tables M[1..n][1..n] and B[1..n-1][2..n]
for i = 1 to n
M[i][i] = 0 // boundary case
for p = 2 to n // p is the chain length
for i = 1 to n – p + 1 // all i, j combinations j = i + p – 1
M[i][j] = ∞
for k = i to j – 1 // find the best k
q = M[i][k] + M[k + 1][j] + l[i - 1] * l[k] * l[j]
if q < M[i][j]
M[i][j] = q
B[i][j] = k // backtracking return M and B
Print-Optimal-Parens(B, i, j) if i == j
print 𝐴𝑖 else
print “(”
Print-Optimal-Parens(B, i, B[i][j]) Print-Optimal-Parens(B, B[i][j] + 1, j)
Exercise
Matrix 𝑨𝟏 𝑨𝟐 𝑨𝟑 𝑨𝟒 𝑨𝟓 𝑨𝟔
Dimension 30 x 35 35 x 15 15 x 5 5 x 10 10 x 20 20 x 25
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
15,750
2,625 750
1,000
5,000 7,875
4,375
2,500
3,500 9,375
7,125
53,75 11,875
10,500 15,125
1 2 3 4 5 6
1 2 3 4 5 6
1
2
3
4
5 1
3
3
5 3
3
3 3
3 3
DP#5: Weighted Interval Scheduling
Textbook Exercise 16.2-2
Interval Scheduling
• Input: 𝑛 job requests with start times 𝑠
𝑖, finish times 𝑓
𝑖• Output: the maximum number of compatible jobs
• The interval scheduling problem can be solved using an “early-finish-time- first” greedy algorithm in 𝑂(𝑛) time
“Greedy Algorithm”Next topic!
time 1
2 3 4 5 6
job index
Weighted Interval Scheduling
• Input: 𝑛 job requests with start times 𝑠
𝑖, finish times 𝑓
𝑖, and values 𝑣
𝑖• Output: the maximum total value obtainable from compatible jobs
1
3
3 4
3 1 1
2 3 4 5 6
job index
Assume that the requests are sorted in non-decreasing order (𝑓𝑖 ≤ 𝑓𝑗 when 𝑖 < 𝑗) 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible
e.g. 𝑝 1 = 0, 𝑝 2 = 0, 𝑝 3 = 1, 𝑝 4 = 1, 𝑝 5 = 4, 𝑝 6 = 3
Step 1: Characterize an OPT Solution
• Subproblems
• WIS(i): weighted interval scheduling for the first 𝑖 jobs
• Goal: WIS(n)
• Optimal substructure: suppose OPT is an optimal solution to WIS(i) , there are 2 cases:
• Case 1: job 𝑖 in OPT
• OPT\{𝑖} is an optimal solution of WIS(p(i))
• Case 2: job 𝑖 not in OPT
• OPT is an optimal solution of WIS(i-1) time
1 3
3 4
3 1 1
2 3 4 5 6
job index
2
1 3 4 5 6 7 8 9
4
1
Weighted Interval Scheduling Problem
Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible
Step 2: Recursively Define the Value of an OPT Solution
• Optimal substructure: suppose OPT is an optimal solution to WIS(i) , there are 2 cases:
• Case 1: job 𝑖 in OPT
• OPT\{𝑖} is an optimal solution of WIS(p(i))
• Case 2: job 𝑖 not in OPT
• OPT is an optimal solution of WIS(i-1)
• Recursively define the value
Weighted Interval Scheduling Problem
Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
i 0 1 2 3 4 5 … n
M[i]
WIS(n, s, f, v, p) M[0] = 0
for i = 1 to n
M[i] = max(v[i] + M[p[i]], M[i - 1]) return M[n]
Weighted Interval Scheduling Problem
Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible
Step 4: Construct an OPT Solution by Backtracking
• Bottom-up method: solve smaller subproblems first
i 0 1 2 3 4 5 6
M[i] 0 1 3 4 5 6 7
1
3
3 4
1 1
2 3 4 5
job index
Weighted Interval Scheduling Problem
Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible
Step 4: Construct an OPT Solution by Backtracking
WIS(n, s, f, v, p) M[0] = 0
for i = 1 to n
M[i] = max(v[i] + M[p[i]], M[i - 1]) return M[n]
Weighted Interval Scheduling Problem
Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible
Find-Solution(M, n) if n = 0
return {}
if v[n] + M[p[n]] > M[n-1] // case 1 return {n} ∪ Find-Solution(p[n]) return Find-Solution(n-1) // case 2
DP#3: Knapsack (背包問題)
Textbook Exercise 16.2-2
Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊
• Variants of knapsack problem
• 0-1 Knapsack Problem: 每項物品只能拿一個
• Unbounded Knapsack Problem: 每項物品可以拿多個
• Multidimensional Knapsack Problem: 背包空間有限
• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
• Fractional Knapsack Problem: 物品可以只拿部分
Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊
• Variants of knapsack problem
• 0-1 Knapsack Problem: 每項物品只能拿一個
• Unbounded Knapsack Problem: 每項物品可以拿多個
• Multidimensional Knapsack Problem: 背包空間有限
• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
• Fractional Knapsack Problem: 物品可以只拿部分
Step 1: Characterize an OPT Solution
• Subproblems
• ZO-KP(i, w): 0-1 knapsack problem within 𝑤 capacity for the first 𝑖 items
• Goal: ZO-KP(n, W)
• Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w) , there are 2 cases:
• Case 1: item 𝑖 in OPT
• OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)
• Case 2: item 𝑖 not in OPT
• OPT is an optimal solution of ZO-KP(i - 1, w)
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
ZO-KP(i) ZO-KP(i, w)
consider the available capacity
Step 2: Recursively Define the Value of an OPT Solution
• Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w) , there are 2 cases:
• Case 1: item 𝑖 in OPT
• OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)
• Case 2: item 𝑖 not in OPT
• OPT is an optimal solution of ZO-KP(i - 1, w)
• Recursively define the value
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
i\w 0 1 2 3 … w … W
0 1 2 i n
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
i\w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 4 4 4 4 4
2 0 4 9 13 13 13
3 0 4 9 13 20 24
i wi vi
1 1 4
2 2 9
3 4 20
𝑊 = 5
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
ZO-KP(n, v, W) for w = 0 to W
M[0, w] = 0 for i = 1 to n
for w = 0 to W if(wi > w)
M[i, w] = M[i-1, w]
else
M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
Step 4: Construct an OPT Solution by Backtracking
ZO-KP(n, v, W) for w = 0 to W
M[0, w] = 0 for i = 1 to n
for w = 0 to W if(wi > w)
M[i, w] = M[i-1, w]
else
M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]
Find-Solution(M, n, W) S = {}
w = W
for i = n to 1
if M[i, w] > M[i – 1, w] // case 1 w = w – wi
S = S ∪ {i}
Pseudo-Polynomial Time
• Polynomial: polynomial in the length of the input (#bits for the input)
• Pseudo-polynomial: polynomial in the numeric value
• The time complexity of 0-1 knapsack problem is Θ 𝑛𝑊
• 𝑛: number of objects
• 𝑊: knapsack’s capacity (non-negative integer)
• polynomial in the numeric value
= pseudo-polynomial in input size
= exponential in the length of the input
• Note: the size of the representation of 𝑊 is log
2𝑊
= 2𝑚 = 𝑚
Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊
• Variants of knapsack problem
• 0-1 Knapsack Problem: 每項物品只能拿一個
• Unbounded Knapsack Problem: 每項物品可以拿多個
• Multidimensional Knapsack Problem: 背包空間有限
• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
• Fractional Knapsack Problem: 物品可以只拿部分
Step 1: Characterize an OPT Solution
• Subproblems
• U-KP(i, w): unbounded knapsack problem with 𝑤 capacity for the first 𝑖 items
• Goal: U-KP(n, W)
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity
0-1 Knapsack Problem Unbounded Knapsack Problem
each item can be chosen at most once each item can be chosen multiple times a sequence of binary choices: whether to
choose item 𝑖
a sequence of 𝑖 choices: which one (from 1 to 𝑖) to choose
Time complexity = Θ 𝑛𝑊 Time complexity = Θ 𝑛2𝑊
Can we do better?
Step 1: Characterize an OPT Solution
• Subproblems
• U-KP(w): unbounded knapsack problem with 𝑤 capacity
• Goal: U-KP(W)
• Optimal substructure: suppose OPT is an optimal solution to
U-KP(w), there are 𝑛 cases:
• Case 1: item 1 in OPT
• Removing an item 1 from OPT is an optimal solution of U-KP(w – w1)
• Case 2: item 2 in OPT
• Removing an item 2 from OPT is an optimal solution of U-KP(w – w2) :
• Case 𝑛: item 𝑛 in OPT
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity
Step 2: Recursively Define the Value of an OPT Solution
• Optimal substructure: suppose OPT is an optimal solution to U-KP(w) , there are 𝑛 cases:
• Case 𝑖: item 𝑖 in OPT
• Removing an item i from OPT is an optimal solution of U-KP(w – w1)
• Recursively define the value
只考慮背包還裝的下的情形
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
w 0 1 2 3 4 5 … W
M[w]
i wi vi
1 1 4
2 2 9
3 4 20
𝑊 = 5
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
w 0 1 2 3 4 5
M[w] 0 4 9 13 18 22
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity
i wi vi
1 1 4
2 2 9
3 4 20
𝑊 = 5
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
U-KP(v, W)
for w = 0 to W M[w] = 0
for w = 0 to W for i = 1 to n
if(wi <= w)
tmp = vi + M[w - wi] M[w] = max(M[w], tmp)
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖 has unlimited supplies Output: the max value within 𝑊 capacity
Step 4: Construct an OPT Solution by Backtracking
U-KP(v, W)
for w = 0 to W M[w] = 0
for w = 0 to W for i = 1 to n
if(wi <= w)
tmp = vi + M[w - wi] M[w] = max(M[w], tmp) return M[W]
Find-Solution(M, n, W) for i = 1 to n
C[i] = 0 // C[i] = # of item i in solution w = W
for i = i to n while w > 0
if(wi <= w && M[w] == (vi + M[w - wi])) w = w - wi
C[i] += 1 return C
Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊
• Variants of knapsack problem
• 0-1 Knapsack Problem: 每項物品只能拿一個
• Unbounded Knapsack Problem: 每項物品可以拿多個
• Multidimensional Knapsack Problem: 背包空間有限
• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
• Fractional Knapsack Problem: 物品可以只拿部分
Step 1: Characterize an OPT Solution
• Subproblems
• M-KP(i, w, d): multidimensional knapsack problem with 𝑤 capacity and 𝑑 size for the first 𝑖 items
• Goal: M-KP(n, W, D)
• Optimal substructure: suppose OPT is an optimal solution to
M-KP(i, w, d), there are 2 cases:
• Case 1: item 𝑖 in OPT
• OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)
• Case 2: item 𝑖 not in OPT
• OPT is an optimal solution of M-KP(i - 1, w, d)
Multidimensional Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖
Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once
Step 2: Recursively Define the Value of an OPT Solution
• Optimal substructure: suppose OPT is an optimal solution to M-KP(i, w, d) , there are 2 cases:
• Case 1: item 𝑖 in OPT
• OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)
• Case 2: item 𝑖 not in OPT
• OPT is an optimal solution of M-KP(i - 1, w, d)
• Recursively define the value
Multidimensional Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖
Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once
Exercise
• Step 3: Compute Value of an OPT Solution
• Step 4: Construct an OPT Solution by Backtracking
• What is the time complexity?
Multidimensional Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖
Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once
Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊
• Variants of knapsack problem
• 0-1 Knapsack Problem: 每項物品只能拿一個
• Unbounded Knapsack Problem: 每項物品可以拿多個
• Multidimensional Knapsack Problem: 背包空間有限
• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
• Fractional Knapsack Problem: 物品可以只拿部分
Multiple-Choice Knapsack Problem
• Input: 𝑛 items
• 𝑣𝑖,𝑗: value of 𝑗-th item in the group 𝑖
• 𝑤𝑖,𝑗: weight of 𝑗-th item in the group 𝑖
• 𝑛𝑖: number of items in group 𝑖
• 𝑛: total number of items (σ 𝑛𝑖)
• 𝐺: total number of groups
• Output: the maximum value for the knapsack with capacity of 𝑊, where the item from each group can be selected at most once
group 2 group 3
Step 1: Characterize an OPT Solution
• Subproblems
• MC-KP(w): 𝑤 capacity
• MC-KP(i, w): 𝑤 capacity for the first 𝑖 groups
• MC-KP(i, j, w): 𝑤 capacity for the first 𝑗 items from first 𝑖 groups Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)
Output: the max value within 𝑊 capacity, where each group is chosen at most once
Which one is more suitable for this problem?
the constraint is for groups
Step 1: Characterize an OPT Solution
• Subproblems
• MC-KP(i, w): multi-choice knapsack problem with 𝑤 capacity for the first 𝑖 groups
• Goal: MC-KP(G, W)
• Optimal substructure: suppose OPT is an optimal solution to MC-KP(i, w) , for the group 𝑖, there are 𝑛
𝑖+ 1 cases:
• Case 1: no item from 𝑖-th group in OPT
• OPT is an optimal solution of MC-KP(i - 1, w)
:
• Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)
Output: the max value within 𝑊 capacity, where each group is chosen at most once
Step 2: Recursively Define the Value of an OPT Solution
• Optimal substructure: suppose OPT is an optimal solution to MC-KP(i, w) , for the group 𝑖, there are 𝑛
𝑖+ 1 cases:
• Case 1: no item from 𝑖-th group in OPT
• OPT is an optimal solution of MC-KP(i - 1, w)
• Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT
• OPT\itemi,j is an optimal solution of MC-KP(i - 1, w – wi,j)
• Recursively define the value
Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)
Output: the max value within 𝑊 capacity, where each group is chosen at most once
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
i\w 0 1 2 3 … w … W
0 1 2 i n
Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)
Output: the max value within 𝑊 capacity, where each group is chosen at most once
Step 3: Compute Value of an OPT Solution
• Bottom-up method: solve smaller subproblems first
MC-KP(n, v, W) for w = 0 to W
M[0, w] = 0
for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w
M[i, w] = M[i - 1, w]
for j = 1 to ni // check j-th item in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])
M[i, w] = vi,j + M[i - 1, w - wi,j] return M[G, W]
Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups)
Output: the max value within 𝑊 capacity, where each group is chosen at most once
Step 4: Construct an OPT Solution by Backtracking
MC-KP(n, v, W) for w = 0 to W
M[0, w] = 0
for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w
M[i, w] = M[i - 1, w]
for j = 1 to ni // check items in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])
M[i, w] = vi,j + M[i - 1, w - wi,j] B[i, w] = j
return M[G, W], B[G, W]
Practice to write the pseudo code for Find-Solution()
Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊
• Variants of knapsack problem
• 0-1 Knapsack Problem: 每項物品只能拿一個
• Unbounded Knapsack Problem: 每項物品可以拿多個
• Multidimensional Knapsack Problem: 背包空間有限
• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
• Fractional Knapsack Problem: 物品可以只拿部分
Fractional Knapsack Problem
• Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
• Output: the maximum value for the knapsack with capacity of 𝑊, where we can take any fraction of items
• Dynamic programming algorithm should work
• Choose maximal
𝑣𝑖𝑤𝑖
(類似CP值) first
“Greedy Algorithm”Next topic!
Can we do better?
Concluding Remarks
• “Dynamic Programming”: solve many subproblems in polynomial time for which a naïve approach would take exponential time
• When to use DP
• Whether subproblem solutions can combine into the original solution
• When subproblems are overlapping
• Whether the problem has optimal substructure
• Common for optimization problem
• Two ways to avoid recomputation
• Top-down with memoization
• Bottom-up method
• Complexity analysis
• Space for tabular filling
• Size of the subproblem graph
Question?
Important announcement will be sent to
@ntu.edu.tw mailbox & post to the course website
Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw