▪
Mini-HW 4 released
▪ Due on 10/18 (Thu) 14:20
▪
Homework 1 due a week later
▪ A4 hardcopy submitted before the class starts
▪ Softcopy submitted to NTU COOL before the deadline
▪
Homework 2 released
▪ Due on 11/06 (Tue) 18:00 (3.5 weeks)
▪ Submitted to NTU COOL only
Frequently check the website for the updated information!
▪ Dynamic Programming
▪ DP #1: Rod Cutting
▪ DP #2: Stamp Problem
▪ DP #3: Knapsack Problem
▪ 0/1 Knapsack
▪ Unbounded Knapsack
▪ Multidimensional Knapsack
▪ Fractional Knapsack
▪ DP #4: Matrix-Chain Multiplication
▪ DP #5: Sequence Alignment Problem
▪ Longest Common Subsequence (LCS) / Edit Distance
▪ Viterbi Algorithm
▪ Space Efficient Algorithm
▪
有100個死囚,隔天執行死刑,典獄長開恩給他們一個存活的機會。
▪
當隔天執行死刑時,每人頭上戴一頂帽子(黑或白)排成一隊伍,在
死刑執行前,由隊伍中最後的囚犯開始,每個人可以猜測自己頭上 的帽子顏色(只允許說黑或白),猜對則免除死刑,猜錯則執行死刑。
▪
若這些囚犯可以前一天晚上先聚集討論方案,是否有好的方法可以
使總共存活的囚犯數量期望值最高?
▪
囚犯排成一排,每個人可以看到前面所有人的帽子,但看不到自己 及後面囚犯的。
▪
由最後一個囚犯開始猜測,依序往前。
▪
每個囚犯皆可聽到之前所有囚犯的猜測內容。
……
Example: 奇數者猜測內 容為前面一位的帽子顏 色 → 存活期望值為75人 有沒有更多人可以存活的好策略?
8
▪
Divide-and-Conquer
▪ partition the problem into independent or disjoint subproblems
▪ repeatedly solving the common subsubproblems
→ more work than necessary
▪
Dynamic Programming
▪ partition the problem into dependent or overlapping subproblems
▪ avoid recomputation
✓Top-down with memoization
✓Bottom-up method
▪
DP procedure
1. Characterize the structure of an optimal solution
2. Recursively define the value of an optimal solution
3. Compute the value of an optimal solution, typically in a bottom-upfashion
4. Construct an optimal solution from computed information
▪
Two key properties of DP for optimization
▪ Overlapping subproblems
▪ Optimal substructure – an optimal solution can be constructed from optimal solutions to subproblems
✓Reduce search space (ignore non-optimal solutions)
Textbook Chapter 15.1 – Rod Cutting
11
▪
Input: a rod of length 𝑛 and a table of prices 𝑝
𝑖for 𝑖 = 1, … , 𝑛
▪
Output: the maximum revenue 𝑟
𝑛obtainable by cutting up the rod and selling the pieces
length 𝑖 (m) 1 2 3 4 5
price 𝑝𝑖 1 5 8 9 10
4m
2m
2m
▪
A rod with the length = 4
length 𝑖 (m) 1 2 3 4 5
price 𝑝𝑖 1 5 8 9 10
4m
3m 1m
2m 2m
1m 3m
2m 1m 1m
1m 2m 1m
1m 2m 1m
1m 1m 1m
1m
→ 9
→ 8 + 1 = 9
→ 5 + 5 = 10
→ 1 + 8 = 9
→ 5 + 1 + 1 = 7
→ 1 + 5 + 1 = 7
→ 1 + 1 + 5 = 7
→ 1 + 1 + 1 + 1 = 4
▪
A rod with the length = 𝑛
▪ For each integer position, we can choose “cut” or “not cut”
▪ There are 𝑛 – 1 positions for consideration
▪
The total number of cutting results is 2
𝑛−1= Θ 2
𝑛−1length 𝑖 (m) 1 2 3 4 5
price 𝑝𝑖 1 5 8 9 10
n
▪
We use a recursive function to solve the subproblems
▪
If we know the answer to the subproblem, can we get the answer to the original problem?
▪
Optimal substructure – an optimal solution can be constructed from optimal solutions to subproblems
𝑟𝑛−𝑖 𝑟𝑖
no cut
cut at the i-th position (from left to right)
𝑟𝑛: the maximum revenue obtainable for a rod of length 𝑛
▪
Version 1
▪
Version 2
▪ try to reduce the number of subproblems → focus on the left-most cut no cut
cut at the i-th position (from left to right)
left-most value maximum value obtainable 𝑟𝑛−𝑖
𝑝𝑖
▪
Focus on the left-most cut
▪ assume that we always cut from left to right → the first cut
optimal solution to subproblems
𝑟𝑛−𝑖 𝑝𝑖
𝑟𝑛−1 𝑝1
𝑟𝑛−2 𝑝2
: : optimal solution
▪
𝑇 𝑛 = time for running
Cut-Rod(p, n) Cut-Rod(p, n)// base case if n == 0
return 0
// recursive case q = -∞
for i = 1 to n
q = max(q, p[i] + Cut-Rod(p, n - i)) return q
▪
Rod cutting problem
Cut-Rod(p, n) // base case if n == 0
return 0
// recursive case q = -∞
for i = 1 to n
q = max(q, p[i] + Cut-Rod(p, n - i)) return q
CR(4)
CR(3) CR(0)
CR(2) CR(1) CR(1) CR(0)
CR(1) CR(0) CR(0)
CR(0)
CR(0)
Calling overlapping subproblems result in poor efficiency
CR(2) CR(1)
CR(0) CR(0)
▪
Idea: use space for better time efficiency
▪
Rod cutting problem has overlapping subproblems and optimal substructures
→ can be solved by DP
▪
When the number of subproblems is polynomial, the time complexity is polynomial using DP
▪
DP algorithm
▪ Top-down: solve overlapping subproblems recursively with memoization
▪ Bottom-up: build up solutions to larger and larger subproblems
▪
Top-Down with Memoization
▪ Solve recursively and memo the subsolutions (跳著填表)
▪ Suitable that not all
subproblems should be solved
▪
Bottom-Up with Tabulation
▪ Fill the table from small to large
▪ Suitable that each small problem should be solved
f(0) f(1) f(2) … f(n) f(0) f(1) f(2) … f(n)
Memoized-Cut-Rod(p, n)
// initialize memo (an array r[] to keep max revenue) r[0] = 0
for i = 1 to n
r[i] = -∞ // r[i] = max revenue for rod with length=i return Memorized-Cut-Rod-Aux(p, n, r)
Memoized-Cut-Rod-Aux(p, n, r) if r[n] >= 0
return r[n] // return the saved solution q = -∞
for i = 1 to n
q = max(q, p[i] + Memoized-Cut-Rod-Aux(p, n-i, r)) r[n] = q // update memo
return q
▪
𝑇 𝑛 = time for running
Memoized-Cut-Rod(p, n)Bottom-Up-Cut-Rod(p, n) r[0] = 0
for j = 1 to n // compute r[1], r[2], ... in order q = -∞
for i = 1 to j
q = max(q, p[i] + r[j - i]) r[j] = q
return r[n]
▪
𝑇 𝑛 = time for running
Bottom-Up-Cut-Rod(p, n)▪
Input: a rod of length 𝑛 and a table of prices 𝑝
𝑖for 𝑖 = 1, … , 𝑛
▪
Output: the maximum revenue 𝑟
𝑛obtainable and the list of cut pieces
length 𝑖 (m) 1 2 3 4 5
price 𝑝𝑖 1 5 8 9 10
4m
2m
2m
▪
Add an array to keep the cutting positions cut
Extended-Bottom-Up-Cut-Rod(p, n) r[0] = 0
for j = 1 to n //compute r[1], r[2], ... in order q = -∞
for i = 1 to j
if q < p[i] + r[j - i]
q = p[i] + r[j - i]
cut[j] = i // the best first cut for len j rod r[i] = q
return r[n], cut
Print-Cut-Rod-Solution(p, n)
(r, cut) = Extended-Bottom-up-Cut-Rod(p, n) while n > 0
print cut[n]
n = n – cut[n] // remove the first piece
f(0) f(1) f(2) … f(n)
▪
Top-Down with Memoization
▪ Better when some subproblems not be solved at all
▪ Solve only the required parts of subproblems
▪
Bottom-Up with Tabulation
▪ Better when all subproblems must be solved at least once
▪ Typically outperform top-down method by a constant factor
▪ No overhead for recursive calls
▪ Less overhead for maintaining the table
f(0) f(1) f(2) … f(n)
F(5) F(4) F(3) F(2) F(1) F(0)
▪ Approach 1: approximate via (#subproblems) * (#choices for each subproblem)
▪
For rod cutting
▪ #subproblems = n
▪ #choices for each subproblem = O(n)
▪ → T(n) is about O(n2)
▪ Approach 2: approximate via subproblem graphs
▪
The size of the subproblem graph allows us to estimate the time complexity of the DP algorithm
▪
A graph illustrates the set of subproblems involved and how subproblems depend on another 𝐺 = 𝑉, 𝐸 (E: edge, V: vertex)
▪ 𝑉 : #subproblems
▪ A subproblem is run only once
▪ |𝐸|: sum of #subsubproblems are needed for each subproblem
▪ Time complexity: linear to 𝑂( 𝐸 + 𝑉 )
F(5) F(4)
F(3)
F(2)
F(1)
F(0)
Bottom-up: Reverse Topological Sort Top-down: Depth First Search
Graph Algorithm (taught later)
1.
Characterize the structure of an optimal solution
✓
Overlapping subproblems: revisit same subproblems
✓
Optimal substructure: an optimal solution to the problem contains within it optimal solutions to subproblems
2.
Recursively define the value of an optimal solution
✓
Express the solution of the original problem in terms of optimal solutions for subproblems
3.
Compute the value of an optimal solution
✓
typically in a bottom-up fashion
4.
Construct an optimal solution from computed information
✓
Step 3 and 4 may be combined
1. Characterize the structure of an optimal solution
2. Recursively define the value of an optimal solution
3. Compute the value of an optimal solution
4. Construct an optimal solution from computed information
▪
Step 1-Q1: What can be the subproblems?
▪
Step 1-Q2: Does it exhibit optimal structure? (an optimal solution can be represented by the optimal solutions to subproblems)
▪ Yes. → continue
▪ No. → go to Step 1-Q1 or there is no DP solution for this problem Rod Cutting Problem
Input: a rod of length 𝑛 and a table of prices 𝑝𝑖 for 𝑖 = 1, … , 𝑛 Output: the maximum revenue 𝑟𝑛 obtainable
▪
Step 1-Q1: What can be the subproblems?
▪ Subproblems: Cut-Rod(0), Cut-Rod(1), …, Cut-Rod(n-1)
▪ Cut-Rod(i): rod cutting problem with length-i rod
▪ Goal: Cut-Rod(n)
▪ Suppose we know the optimal solution to Cut-Rod(i), there are i cases:
▪ Case 1: the first segment in the solution has length 1
▪ Case 2: the first segment in the solution has length 2
:
▪ Case i: the first segment in the solution has length i Rod Cutting Problem
Input: a rod of length 𝑛 and a table of prices 𝑝𝑖 for 𝑖 = 1, … , 𝑛 Output: the maximum revenue 𝑟𝑛 obtainable
從solution中拿掉一段長度為1的鐵條, 剩下的部分是Cut-Rod(i-1)的最佳解 從solution中拿掉一段長度為2的鐵條, 剩下的部分是Cut-Rod(i-2)的最佳解
▪
Step 1-Q2: Does it exhibit optimal structure? (an optimal solution can be represented by the optimal solutions to subproblems)
▪
Yes. Prove by contradiction.
Rod Cutting Problem
Input: a rod of length 𝑛 and a table of prices 𝑝𝑖 for 𝑖 = 1, … , 𝑛 Output: the maximum revenue 𝑟𝑛 obtainable
▪ Suppose we know the optimal solution to Cut-Rod(i), there are i cases:
▪ Case 1: the first segment in the solution has length 1
▪ Case 2: the first segment in the solution has length 2
:
▪ Case i: the first segment in the solution has length i
▪ Recursively define the value Rod Cutting Problem
Input: a rod of length 𝑛 and a table of prices 𝑝𝑖 for 𝑖 = 1, … , 𝑛 Output: the maximum revenue 𝑟𝑛 obtainable
從solution中拿掉一段長度為1的鐵條, 剩下的部分是Cut-Rod(i-1)的最佳解 從solution中拿掉一段長度為2的鐵條, 剩下的部分是Cut-Rod(i-2)的最佳解
從solution中拿掉一段長度為i的鐵條, 剩下的部分是Cut-Rod(0)的最佳解
▪ Bottom-up method: solve smaller subproblems first
35
Rod Cutting Problem
Input: a rod of length 𝑛 and a table of prices 𝑝𝑖 for 𝑖 = 1, … , 𝑛 Output: the maximum revenue 𝑟𝑛 obtainable
i 0 1 2 3 4 5 … n
r[i]
Bottom-Up-Cut-Rod(p, n) r[0] = 0
for j = 1 to n // compute r[1], r[2], ... in order q = -∞
for i = 1 to j
q = max(q, p[i] + r[j - i]) r[j] = q
▪ Bottom-up method: solve smaller subproblems first Rod Cutting Problem
Input: a rod of length 𝑛 and a table of prices 𝑝𝑖 for 𝑖 = 1, … , 𝑛 Output: the maximum revenue 𝑟𝑛 obtainable
i 0 1 2 3 4 5 … n
r[i] 0
cut[i] 0 1
1
2 5
3 8
2 10
length 𝑖 1 2 3 4 5
price 𝑝𝑖 1 5 8 9 10
Cut-Rod(p, n) r[0] = 0
for j = 1 to n // compute r[1], r[2], ... in order q = -∞
for i = 1 to j
if q < p[i] + r[j - i]
q = p[i] + r[j - i]
cut[j] = i // the best first cut for len j rod r[i] = q
return r[n], cut
Print-Cut-Rod-Solution(p, n) (r, cut) = Cut-Rod(p, n) while n > 0
print cut[n]
n = n – cut[n] // remove the first piece
38
▪ Input: the postage 𝑛 and the stamps with values 𝑣
1, 𝑣
2, … , 𝑣
𝑘▪ Output: the minimum number of stamps to cover the postage
▪
The optimal solution 𝑆
𝑛can be recursively defined as
Stamp(v, n) r_min = ∞
if n == 0 // base case return 0
for i = 1 to k // recursive case r[i] = Stamp(v, n - v[i])
if r[i] < r_min r_min = r[i]
return r_min + 1
▪
Subproblems
▪ S(i): the min #stamps with postage i
▪ Goal: S(n)
▪
Optimal substructure: suppose we know the optimal solution to S(i), there are k cases:
▪ Case 1: there is a stamp with v1 in OPT
▪ Case 2: there is a stamp with v2 in OPT
:
▪ Case k: there is a stamp with vk in OPT
41
Stamp Problem
Input: the postage 𝑛 and the stamps with values 𝑣1, 𝑣2, … , 𝑣𝑘 Output: the minimum number of stamps to cover the postage
從solution中拿掉一張郵資為v1的郵票, 剩下的部分是S(i-v[1])的最佳解 從solution中拿掉一張郵資為v2的郵票, 剩下的部分是S(i-v[2])的最佳解
▪ Suppose we know the optimal solution to S(i), there are k cases:
▪ Case 1: there is a stamp with v1 in OPT
▪ Case 2: there is a stamp with v2 in OPT
:
▪ Case k: there is a stamp with vk in OPT
▪
Recursively define the value
Stamp Problem
Input: the postage 𝑛 and the stamps with values 𝑣1, 𝑣2, … , 𝑣𝑘 Output: the minimum number of stamps to cover the postage
從solution中拿掉一張郵資為v1的郵票, 剩下的部分是S(i-v[1])的最佳解 從solution中拿掉一張郵資為v2的郵票, 剩下的部分是S(i-v[2])的最佳解
從solution中拿掉一張郵資為vk的郵票, 剩下的部分是S(i-v[k])的最佳解
▪ Bottom-up method: solve smaller subproblems first
43
Stamp Problem
Input: the postage 𝑛 and the stamps with values 𝑣1, 𝑣2, … , 𝑣𝑘 Output: the minimum number of stamps to cover the postage
i 0 1 2 3 4 5 … n
S[i]
Stamp(v, n) S[0] = 0
for i = 1 to n // compute r[1], r[2], ... in order r_min = ∞
for j = 1 to k
if S[i - v[j]] < r_min r_min = 1 + S[i – v[j]]
S[i] = r_min
Stamp(v, n) S[0] = 0
for i = 1 to n r_min = ∞
for j = 1 to k
if S[i - v[j]] < r_min r_min = 1 + S[i – v[j]]
B[i] = j // backtracking for stamp with v[j]
S[i] = r_min return S[n], B
Print-Stamp-Selection(v, n) (S, B) = Stamp(v, n)
while n > 0 print B[n]
n = n – v[B[n]]
Textbook Exercise 16.2-2
45
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊
▪
Variants of knapsack problem
▪ 0-1 Knapsack Problem: 每項物品只能拿一個
▪ Unbounded Knapsack Problem: 每項物品可以拿多個
▪ Multidimensional Knapsack Problem: 背包空間有限
▪ Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
▪ Fractional Knapsack Problem: 物品可以只拿部分
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊
▪
Variants of knapsack problem
▪ 0-1 Knapsack Problem: 每項物品只能拿一個
▪ Unbounded Knapsack Problem: 每項物品可以拿多個
▪ Multidimensional Knapsack Problem: 背包空間有限
▪ Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
▪ Fractional Knapsack Problem: 物品可以只拿部分
▪
Subproblems
▪ ZO-KP(i, w): 0-1 knapsack problem within 𝑤 capacity for the first 𝑖 items
▪ Goal: ZO-KP(n, W)
▪
Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w), there are 2 cases:
▪ Case 1: item 𝑖 in OPT
▪ OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)
▪ Case 2: item 𝑖 not in OPT 0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once ZO-KP(i) ZO-KP(i, w)
consider the available capacity
▪
Optimal substructure: suppose OPT is an optimal solution to ZO-KP(i, w), there are 2 cases:
▪ Case 1: item 𝑖 in OPT
▪ OPT\{𝑖} is an optimal solution of ZO-KP(i - 1, w - wi)
▪ Case 2: item 𝑖 not in OPT
▪ OPT is an optimal solution of ZO-KP(i - 1, w)
▪
Recursively define the value
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
▪ Bottom-up method: solve smaller subproblems first
i\w 0 1 2 3 … w … W
0 1 2 i
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
▪ Bottom-up method: solve smaller subproblems first
i\w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 4 4 4 4 4
2 0 4 9 13 13 13
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
i wi vi
1 1 4
2 2 9
3 4 20
𝑊 = 5
▪ Bottom-up method: solve smaller subproblems first
ZO-KP(n, v, W) for w = 0 to W
M[0, w] = 0 for i = 1 to n
for w = 0 to W if(wi > w)
M[i, w] = M[i-1, w]
else
0-1 Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖
Output: the max value within 𝑊 capacity, where each item is chosen at most once
ZO-KP(n, v, W) for w = 0 to W
M[0, w] = 0 for i = 1 to n
for w = 0 to W if(wi > w)
M[i, w] = M[i-1, w]
else
M[i, w] = max(vi + M[i-1, w-wi], M[i-1, w]) return M[n, W]
Find-Solution(M, n, W) S = {}
w = W
for i = n to 1
if M[i, w] > M[i – 1, w] // case 1 w = w – wi
S = S ∪ {i}
▪
Polynomial: polynomial in the length of the input (#bits for the input)
▪
Pseudo-polynomial: polynomial in the numeric value
▪
The time complexity of 0-1 knapsack problem is Θ 𝑛𝑊
▪ 𝑛: number of objects
▪ 𝑊: knapsack’s capacity (non-negative integer)
▪ polynomial in the numeric value
= pseudo-polynomial in input size
= exponential in the length of the input
▪
Note: the size of the representation of 𝑊 is log
2𝑊
= 2𝑚 = 𝑚
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊
▪
Variants of knapsack problem
▪ 0-1 Knapsack Problem: 每項物品只能拿一個
▪ Unbounded Knapsack Problem: 每項物品可以拿多個
▪ Multidimensional Knapsack Problem: 背包空間有限
▪ Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
▪ Fractional Knapsack Problem: 物品可以只拿部分
▪
Subproblems
▪ U-KP(i, w): unbounded knapsack problem with 𝑤 capacity for the first 𝑖 items
▪ Goal: U-KP(n, W)
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖, each has unlimited supplies Output: the max value within 𝑊 capacity
0-1 Knapsack Problem Unbounded Knapsack Problem
each item can be chosen at most once each item can be chosen multiple times a sequence of binary choices: whether
to choose item 𝑖
a sequence of 𝑖 choices: which one (from 1 to 𝑖) to choose
Time complexity = Θ 𝑛𝑊 Time complexity = Θ 𝑛2𝑊
▪ Subproblems
▪ U-KP(w): unbounded knapsack problem with 𝑤 capacity
▪ Goal: U-KP(W)
▪ Optimal substructure: suppose OPT is an optimal solution to U-KP(w), there are 𝑛 cases:
▪ Case 1: item 1 in OPT
▪ Removing an item 1 from OPT is an optimal solution of U-KP(w – w1)
▪ Case 2: item 2 in OPT
▪ Removing an item 2 from OPT is an optimal solution of U-KP(w – w2) :
▪ Case 𝑛: item 𝑛 in OPT
▪ Removing an item 𝑛 from OPT is an optimal solution of U-KP(w - wn)
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖, each has unlimited supplies Output: the max value within 𝑊 capacity
▪
Optimal substructure: suppose OPT is an optimal solution to U-KP(w), there are 𝑛 cases:
▪ Case 𝑖: item 𝑖 in OPT
▪ Removing an item i from OPT is an optimal solution of U-KP(w – w1)
▪
Recursively define the value
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖, each has unlimited supplies Output: the max value within 𝑊 capacity
只考慮背包還裝的下的情形
▪ Bottom-up method: solve smaller subproblems first Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖, each has unlimited supplies Output: the max value within 𝑊 capacity
w 0 1 2 3 4 5 … W
M[w]
i wi vi
1 1 4
2 2 9
3 4 20
𝑊 = 5
▪ Bottom-up method: solve smaller subproblems first Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖, each has unlimited supplies Output: the max value within 𝑊 capacity
w 0 1 2 3 4 5
M[w] 0
i wi vi
1 1 4
2 2 9
3 4 17
𝑊 = 5
4 9 13 18 22
▪ Bottom-up method: solve smaller subproblems first
U-KP(v, W)
for w = 0 to W M[w] = 0 for w = 0 to W
for i = 1 to n if(wi <= w)
tmp = vi + M[w - wi] M[w] = max(M[w], tmp)
Unbounded Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖, each has unlimited supplies Output: the max value within 𝑊 capacity
U-KP(v, W)
for w = 0 to W M[w] = 0
for w = 0 to W for i = 1 to n
if(wi <= w)
tmp = vi + M[w - wi] M[w] = max(M[w], tmp) return M[W]
Find-Solution(M, n, W) for i = 1 to n
C[i] = 0 // C[i] = # of item i in solution w = W
for i = i to n while w > 0
if(wi <= w && M[w] == (vi + M[w - wi])) w = w - wi
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊
▪
Variants of knapsack problem
▪ 0-1 Knapsack Problem: 每項物品只能拿一個
▪ Unbounded Knapsack Problem: 每項物品可以拿多個
▪ Multidimensional Knapsack Problem: 背包空間有限
▪ Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
▪ Fractional Knapsack Problem: 物品可以只拿部分
▪
Subproblems
▪ M-KP(i, w, d): multidimensional knapsack problem with 𝑤 capacity and 𝑑 size for the first 𝑖 items
▪ Goal: M-KP(n, W, D)
▪
Optimal substructure: suppose OPT is an optimal solution to M-KP(i, w, d) , there are 2 cases:
▪ Case 1: item 𝑖 in OPT
▪ OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)
▪ Case 2: item 𝑖 not in OPT
Multidimensional Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖
Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once
▪
Optimal substructure: suppose OPT is an optimal solution to M-KP(i, w, d), there are 2 cases:
▪ Case 1: item 𝑖 in OPT
▪ OPT\{𝑖} is an optimal solution of M-KP(i - 1, w - wi, d – di)
▪ Case 2: item 𝑖 not in OPT
▪ OPT is an optimal solution of M-KP(i - 1, w, d)
▪
Recursively define the value
Multidimensional Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖
Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once
▪ Step 3: Compute Value of an OPT Solution
▪ Step 4: Construct an OPT Solution by Backtracking
▪ What is the time complexity?
Multidimensional Knapsack Problem
Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖, weighs 𝑤𝑖, and size 𝑑𝑖
Output: the max value within 𝑊 capacity and with the size of 𝑫, where each item is chosen at most once
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊
▪
Variants of knapsack problem
▪ 0-1 Knapsack Problem: 每項物品只能拿一個
▪ Unbounded Knapsack Problem: 每項物品可以拿多個
▪ Multidimensional Knapsack Problem: 背包空間有限
▪ Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
▪ Fractional Knapsack Problem: 物品可以只拿部分
▪
Input: 𝑛 items
▪ 𝑣𝑖,𝑗: value of 𝑗-th item in the group 𝑖
▪ 𝑤𝑖,𝑗: weight of 𝑗-th item in the group 𝑖
▪ 𝑛𝑖: number of items in group 𝑖
▪ 𝑛: total number of items (σ 𝑛𝑖)
▪ 𝐺: total number of groups
▪
Output: the maximum value for the knapsack with capacity of 𝑊,
where the item from each group can be selected at most once
▪
Subproblems
▪ MC-KP(w): 𝑤 capacity
▪ MC-KP(i, w): 𝑤 capacity for the first 𝑖 groups
▪ MC-KP(i, j, w): 𝑤 capacity for the first 𝑗 items from first 𝑖 groups Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups) Output: the max value within 𝑊 capacity, where each group is chosen at most once
Which one is more suitable for this problem?
the constraint is for groups
▪
Subproblems
▪ MC-KP(i, w): multi-choice knapsack problem with 𝑤 capacity for the first 𝑖 groups
▪ Goal: MC-KP(G, W)
▪
Optimal substructure: suppose OPT is an optimal solution to MC-KP(i, w), for the group 𝑖, there are 𝑛
𝑖+ 1 cases:
▪ Case 1: no item from 𝑖-th group in OPT
▪ OPT is an optimal solution of MC-KP(i - 1, w)
:
▪ Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups) Output: the max value within 𝑊 capacity, where each group is chosen at most once
▪
Optimal substructure: suppose OPT is an optimal solution to MC-KP(i, w), for the group 𝑖, there are 𝑛
𝑖+ 1 cases:
▪ Case 1: no item from 𝑖-th group in OPT
▪ OPT is an optimal solution of MC-KP(i - 1, w)
▪ Case 𝑗 + 1: 𝑗-th item from 𝑖-th group (itemi,j) in OPT
▪ OPT\itemi,j is an optimal solution of MC-KP(i - 1, w – wi,j)
▪
Recursively define the value
71
Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups) Output: the max value within 𝑊 capacity, where each group is chosen at most once
▪ Bottom-up method: solve smaller subproblems first
i\w 0 1 2 3 … w … W
0 1 2 i
Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups) Output: the max value within 𝑊 capacity, where each group is chosen at most once
▪ Bottom-up method: solve smaller subproblems first
MC-KP(n, v, W) for w = 0 to W
M[0, w] = 0
for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w
M[i, w] = M[i - 1, w]
for j = 1 to ni // check j-th item in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])
M[i, w] = vi,j + M[i - 1, w - wi,j] return M[G, W]
Multiple-Choice Knapsack Problem
Input: 𝑛 items with value 𝑣𝑖,𝑗 and weighs 𝑤𝑖,𝑗 (𝑛𝑖: #items in group 𝑖, 𝐺: #groups) Output: the max value within 𝑊 capacity, where each group is chosen at most once
MC-KP(n, v, W) for w = 0 to W
M[0, w] = 0
for i = 1 to G // consider groups 1 to i for w = 0 to W // consider capacity = w
M[i, w] = M[i - 1, w]
for j = 1 to ni // check items in group i if(vi,j + M[i - 1, w - wi,j] > M[i, w])
M[i, w] = vi,j + M[i - 1, w - wi,j] B[i, w] = j
return M[G, W], B[G, W]
Practice to write the pseudo code for Find-Solution()
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊
▪
Variants of knapsack problem
▪ 0-1 Knapsack Problem: 每項物品只能拿一個
▪ Unbounded Knapsack Problem: 每項物品可以拿多個
▪ Multidimensional Knapsack Problem: 背包空間有限
▪ Multiple-Choice Knapsack Problem: 每一類物品最多拿一個
▪ Fractional Knapsack Problem: 物品可以只拿部分
▪
Input: 𝑛 items where 𝑖-th item has value 𝑣
𝑖and weighs 𝑤
𝑖(𝑣
𝑖and 𝑤
𝑖are positive integers)
▪
Output: the maximum value for the knapsack with capacity of 𝑊, where we can take any fraction of items
▪
Dynamic programming algorithm should work
▪
Choose maximal
𝑣𝑖𝑤𝑖
(類似CP值) first
“Greedy Algorithm”Next topic!
Can we do better?
77
Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw
78
Important announcement will be sent to @ntu.edu.tw mailbox
& post to the course website