Greedy Algorithms
Greedy #1: Activity-Selection / Interval Scheduling
Greedy #2: Coin Changing
Greedy #3: Fractional Knapsack Problem
Greedy #4: Breakpoint Selection
Greedy #5: Huffman Codes
Greedy #6: Task-Scheduling
Greedy #7: Scheduling to Minimize Lateness
2
Do not focus on “specific algorithms”
But “some strategies” to “design” algorithms
First Skill: Divide-and-Conquer (各個擊破)
Second Skill: Dynamic Programming (動態規劃)
Third Skill: Greedy (貪婪法則)
Textbook Chapter 16 – Greedy Algorithms
Textbook Chapter 16.2 – Elements of the greedy strategy
4
always makes the choice that looks best at the moment
makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution
not always yield optimal solution; may end up at local optimal
Greedy: move towards max gradient and hope it is global maximum local maximal
global maximal
local maximal
Dynamic Programming
has optimal substructure
make an informed choice after getting optimal solutions to subproblems
dependent or overlapping subproblems
Greedy Algorithms
has optimal substructure
make a greedy choice before solving the subproblem
no overlapping subproblems
Each round selects only one subproblem
The subproblem size decreases
6
Optimal Solution
Possible Case 1 Possible
Case 2
Possible Case k
max /min
Subproblem Solution Subproblem
Solution
Subproblem Solution
+ +
+
=
Optimal Solution
Greedy Choice
Subproblem Solution
= +
1. Cast the optimization problem as one in which we make a choice and remain one subproblem to solve
2. Demonstrate the optimal substructure
Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem
3. Prove that there is always an optimal solution to the
original problem that makes the greedy choice
To yield an optimal solution, the problem should exhibit
1. Optimal Substructure : an optimal solution to the problem contains within it optimal solutions to subproblems
2. Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal solution
8
Optimal Substructure : an optimal solution to the problem contains within it optimal solutions to subproblems
Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal solution
Show that it exists an optimal solution that “contains” the greedy choice using exchange argument
For any optimal solution OPT, the greedy choice 𝑔 has two cases
𝑔 is in OPT: done
𝑔 not in OPT: modify OPT into OPT’ s.t. OPT’ contains 𝑔 and is at least as good as OPT
OPT OPT’
𝑔
If OPT’ is better than OPT, the property is proved by contradiction
If OPT’ is as good as OPT, then we showed that there exists an optimal solution containing 𝑔 by construction
Textbook Chapter 16.1 – An activity-selection problem
10
Input: 𝑛 activities with start times 𝑠
𝑖and finish times 𝑓
𝑖(the activities are sorted in monotonically increasing order of finish time 𝑓
1≤ 𝑓
2≤ ⋯ ≤ 𝑓
𝑛)
Output: the maximum number of compatible activities
Without loss of generality: 𝑠
1< 𝑠
2< ⋯ < 𝑠
𝑛and 𝑓
1< 𝑓
2< ⋯ < 𝑓
𝑛 大的包小的則不考慮大的 用小的取代大的一定不會變差
1 2 3 4 5 6
activity index
Subproblems
WIS(i): weighted interval scheduling for the first 𝑖 jobs
Goal: WIS(n)
Dynamic programming algorithm
12
i 0 1 2 3 4 5 … n
M[i]
Weighted Interval Scheduling Problem
Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible
Set 𝑣𝑖 = 1 for all 𝑖 to formulate it into the activity-selection problem
Dynamic programming
Optimal substructure is already proved
Greedy algorithm
Activity-Selection Problem
Input: 𝑛 activities with 𝑠𝑖, 𝑓𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible Output: the maximum number of activities
select the 𝑖-th activity
Why does the 𝑖-th activity must appear in an OPT?
Goal:
Proof
Assume there is an OPT solution for the first 𝑖 − 1 activities (𝑀𝑖−1)
𝐴𝑗 is the last activity in the OPT solution
Replacing 𝐴𝑗 with 𝐴𝑖 does not make the OPT worse
time 14
1 2 : i i - 1
:
activity index
1 2 3 4 5 6 7 8 9
Act-Select(n, s, f, v, p) M[0] = 0
for i = 1 to n if p[i] >= 0
M[i] = 1 + M[p[i]]
return M[n]
Find-Solution(M, n) if n = 0
return {}
return {n} ∪ Find-Solution(p[n])
Activity-Selection Problem
Input: 𝑛 activities with 𝑠𝑖, 𝑓𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible Output: the maximum number of activities
Textbook Exercise 16.1
16
Input: 𝑛 dollars and unlimited coins with values 𝑣
𝑖(1, 5, 10, 50)
Output: the minimum number of coins with the total value 𝑛
Cashier’s algorithm: at each iteration, add the coin with the largest value no more than the current total
Does this algorithm return the OPT?
Subproblems
C(i): minimal number of coins for the total value 𝑖
Goal: C(n)
18
Coin Changing Problem
Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛
Suppose OPT is an optimal solution to C(i), there are 4 cases:
Case 1: coin 1 in OPT
OPT\coin1 is an optimal solution of C(i – v1)
Case 2: coin 2 in OPT
OPT\coin2 is an optimal solution of C(i – v2)
Case 3: coin 3 in OPT
OPT\coin3 is an optimal solution of C(i – v3)
Case 4: coin 4 in OPT
OPT\coin4 is an optimal solution of C(i – v4)
Coin Changing Problem
Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛
Greedy choice: select the coin with the largest value no more than the current total
Proof via contradiction (use the case 10 ≤ 𝑖 < 50 for demo)
Assume that there is no OPT including this greedy choice (choose 10)
all OPT use 1, 5, 50 to pay 𝑖
50 cannot be used
#coins with value 5 < 2 otherwise we can use a 10 to have a better output
#coins with value 1 < 5 otherwise we can use a 5 to have a better output
We cannot pay 𝑖 with the constraints (at most 5 + 4 = 9)
20
Coin Changing Problem
Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛
21
Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw
22
Important announcement will be sent to @ntu.edu.tw mailbox
& post to the course website