**Algorithm Design and Analysis** **Greedy Algorithm (1)**

**http://ada.miulab.tw**

**Outline**

### • Greedy Algorithms

### • Greedy #1: Activity-Selection / Interval Scheduling

### • Greedy #2: Coin Changing

### • Greedy #3: Fractional Knapsack Problem

### • Greedy #4: Breakpoint Selection

### • Greedy #5: Huffman Codes

### • Greedy #6: Task-Scheduling

### • Greedy #7: Scheduling to Minimize Lateness

**Algorithm Design Strategy**

### • Do not focus on “specific algorithms”

### • But “some strategies” to “design” algorithms

### • First Skill: Divide-and-Conquer (各個擊破/分治)

### • Second Skill: Dynamic Programming (動態規劃)

### • Third Skill: Greedy (貪婪法則)

**Greedy Algorithms**

Textbook Chapter 16 – Greedy Algorithms

Textbook Chapter 16.2 – Elements of the greedy strategy

**What is Greedy Algorithms?**

### • always makes the choice that looks best at the moment

### • makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution

• not always yield optimal solution; may end up at local optimal

Greedy: move towards max gradient and hope it is global maximum local maximal

global maximal

local maximal

**Algorithm Design Paradigms**

### • Dynamic Programming

**• has optimal substructure**

• make an informed choice after getting optimal solutions to subproblems

**• dependent or overlapping**
subproblems

### • Greedy Algorithms

**• has optimal substructure**

• make a greedy choice before solving the subproblem

**• no overlapping subproblems**

✓ Each round selects only one subproblem

✓ The subproblem size decreases

Optimal Solution

Possible Case 1 Possible

Case 2

Possible Case k

max /min

Subproblem Solution Subproblem

Solution

Subproblem Solution

+

+

+

= ^{Optimal }

Solution

Greedy Choice

Subproblem Solution

= +

**Greedy Procedure**

### 1. Cast the optimization problem as one in which we make a choice and remain one subproblem to solve

### 2. Demonstrate the optimal substructure

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem

### 3. Prove that there is always an optimal solution to the original problem that

### makes the greedy choice

**Greedy Algorithms**

### To yield an optimal solution, the problem should exhibit

### 1. Optimal Substructure : an optimal solution to the problem contains within its optimal solutions to subproblems

### 2. Greedy-Choice Property : making locally optimal (greedy) choices leads to

### a globally optimal solution

**Proof of Correctness Skills**

### • Optimal Substructure : an optimal solution to the problem contains within it optimal solutions to subproblems

### • Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal solution

• Show that it exists an optimal solution that “contains” the greedy choice using
**exchange argument**

• For any optimal solution OPT, the greedy choice 𝑔 has two cases

• 𝑔 is in OPT: done

• 𝑔 not in OPT: modify OPT into OPT’ s.t. OPT’ contains 𝑔 and is at least as good as OPT

OPT OPT’

𝑔

✓ If OPT’ is better than OPT, the property is proved by contradiction

✓ If OPT’ is as good as OPT, then we showed that there exists an optimal solution containing 𝑔 by construction

**Activity-Selection ** **/ Interval Scheduling**

Textbook Chapter 16.1 – An activity-selection problem

**Activity-Selection/ Interval Scheduling**

### • Input: 𝑛 activities with start times 𝑠

_{𝑖}

### and finish times 𝑓

_{𝑖}

### (the activities are sorted in monotonically increasing order of finish time 𝑓

_{1}

### ≤ 𝑓

_{2}

### ≤ ⋯ ≤ 𝑓

_{𝑛}

### )

### • Output: the maximum number of compatible activities

### • Without loss of generality: 𝑠

_{1}

### < 𝑠

_{2}

### < ⋯ < 𝑠

_{𝑛}

### and 𝑓

_{1}

### < 𝑓

_{2}

### < ⋯ < 𝑓

_{𝑛}

• 大的包小的則不考慮大的 → 用小的取代大的一定不會變差

time 1

2 3 4 5 6

activity index

**Weighted Interval Scheduling**

### • Subproblems

• WIS(i): weighted interval scheduling for the first 𝑖 jobs

• Goal: WIS(n)

### • Dynamic programming algorithm

**i** **0** **1** **2** **3** **4** **5** **…** **n**

M[i]

Set 𝑣_{𝑖} = 1 for all 𝑖 to formulate it into the activity-selection problem

**Weighted Interval Scheduling Problem**

Input: 𝑛 jobs with 𝑠_{𝑖}, 𝑓_{𝑖}, 𝑣_{𝑖} , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible
Output: the maximum total value obtainable from compatible

**Activity-Selection Problem**

### • Dynamic programming

**• Optimal substructure is already proved**

### • Greedy algorithm

select the 𝑖-th activity

Why does the 𝑖-th activity must appear

in an OPT?

**Activity-Selection Problem**

Input: 𝑛 activities with 𝑠_{𝑖}, 𝑓_{𝑖} , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible
Output: the maximum number of activities

**Greedy-Choice Property**

### • Goal:

### • Proof

• Assume there is an OPT solution for the first 𝑖 − 1 activities (𝑀_{𝑖−1})

• 𝐴_{𝑗} is the last activity in the OPT solution →

• Replacing 𝐴_{𝑗} with 𝐴_{𝑖} does not make the OPT worse

time 1

2 :

i
*i - 1*

:

activity index

1 2 3 4 5 6 7 8 9

**Pseudo Code**

Act-Select(n, s, f, v, p) M[0] = 0

for i = 1 to n if p[i] >= 0

M[i] = 1 + M[p[i]]

return M[n]

Find-Solution(M, n) if n = 0

return {}

return {n} ∪ Find-Solution(p[n])

**Activity-Selection Problem**

Input: 𝑛 activities with 𝑠_{𝑖}, 𝑓_{𝑖} , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible
Output: the maximum number of activities

**Coin Changing**

Textbook Exercise 16.1

**Coin Changing Problem**

### • Input: 𝑛 dollars and unlimited coins with values 𝑣

_{𝑖}

### (1, 5, 10, 50)

### • Output: the minimum number of coins with the total value 𝑛

**• Cashier’s algorithm: at each iteration, add the coin with the largest value no ** more than the current total

Does this algorithm return the OPT?

**Step 1: Cast Optimization Problem**

### • Subproblems

• C(i): minimal number of coins for the total value 𝑖

• Goal: C(n)

**Coin Changing Problem**

Input: 𝑛 dollars and unlimited coins with values 𝑣_{𝑖} (1, 5, 10, 50)
Output: the minimum number of coins with the total value 𝑛

**Step 2: Prove Optimal Substructure**

### • Suppose OPT is an optimal solution to C(i) , there are 4 cases:

• Case 1: coin 1 in OPT

• OPT\coin1 is an optimal solution of C(i – v_{1})

• Case 2: coin 2 in OPT

• OPT\coin2 is an optimal solution of C(i – v_{2})

• Case 3: coin 3 in OPT

• OPT\coin3 is an optimal solution of C(i – v_{3})

• Case 4: coin 4 in OPT

• OPT\coin4 is an optimal solution of C(i – v_{4})

**Coin Changing Problem**

Input: 𝑛 dollars and unlimited coins with values 𝑣_{𝑖} (1, 5, 10, 50)
Output: the minimum number of coins with the total value 𝑛

**Step 3: Prove Greedy-Choice Property **

### • Greedy choice: select the coin with the largest value no more than the current total

### • Proof via contradiction (use the case 10 ≤ 𝑖 < 50 for demo)

• Assume that there is no OPT including this greedy choice (choose 10)

→ all OPT use 1, 5, 50 to pay 𝑖

• 50 cannot be used

• #coins with value 5 < 2 → otherwise we can use a 10 to have a better output

• #coins with value 1 < 5 → otherwise we can use a 5 to have a better output

• We cannot pay 𝑖 with the constraints (at most 5 + 4 = 9)
**Coin Changing Problem**

Input: 𝑛 dollars and unlimited coins with values 𝑣_{𝑖} (1, 5, 10, 50)
Output: the minimum number of coins with the total value 𝑛

**To Be Continued…**

## Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw