• 沒有找到結果。

# Algorithm Design and Analysis Greedy Algorithms (1)

N/A
N/A
Protected

Share "Algorithm Design and Analysis Greedy Algorithms (1)"

Copied!
65
0
0

(1)

(2)

(3)

(4)

## Greedy Algorithms

Textbook Chapter 16 – Greedy Algorithms

Textbook Chapter 16.2 – Elements of the greedy strategy

(5)

### • makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution

• not always yield optimal solution; may end up at local optimal

local maximal

global maximal

local maximal

(6)

### • Dynamic Programming

• has optimal substructure

• make an informed choice after getting optimal solutions to subproblems

• dependent or overlapping subproblems

### • Greedy Algorithms

• has optimal substructure

• make a greedy choice before solving the subproblem

• no overlapping subproblems

✓ Each round selects only one subproblem

✓ The subproblem size decreases

Optimal Solution

Possible Case 1 Possible

Case 2

max /min

Subproblem Solution Subproblem

Solution

+

= + Optimal

Solution

Greedy Choice

Subproblem Solution

= +

(7)

### 2. Demonstrate the optimal substructure

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem

(8)

(9)

### • Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal solution

• Show that it exists an optimal solution that “contains” the greedy choice using exchange argument

• For any optimal solution OPT, the greedy choice 𝑔 has two cases

• 𝑔 is in OPT: done

• 𝑔 not in OPT: modify OPT into OPT’ s.t. OPT’ contains 𝑔 and is at least as good as OPT

OPT OPT’

If OPT’ is better than OPT, the property is proved by contradiction

If OPT’ is as good as OPT, then we showed that there exists an optimal solution containing 𝑔 by construction

(10)

## Greedy #1: Activity-Selection / Interval Scheduling

Textbook Chapter 16.1 – An activity-selection problem

(11)

𝑖

𝑖

1

2

𝑛

1

2

𝑛

1

2

### < ⋯ < 𝑓

𝑛

• 大的包小的則不考慮大的 → 用小的取代大的一定不會變差

1 2 3 4 5 6

activity index

(12)

### • Subproblems

• WIS(i): weighted interval scheduling for the first 𝑖 jobs

• Goal: WIS(n)

### • Dynamic programming algorithm

i 0 1 2 3 4 5 n

M[i]

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

(13)

### • Dynamic programming

• Optimal substructure is already proved

### • Greedy algorithm

select the 𝑖-th activity

Why does the 𝑖-th activity must appear

in an OPT?

Activity-Selection Problem

Input: 𝑛 activities with 𝑠𝑖, 𝑓𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible Output: the maximum number of activities

(14)

### • Proof

• Assume there is an OPT solution for the first 𝑖 − 1 activities (𝑀𝑖−1)

• 𝐴𝑗 is the last activity in the OPT solution →

• Replacing 𝐴𝑗 with 𝐴𝑖 does not make the OPT worse

1 2 : i i - 1

activity index

(15)

### Pseudo Code

Act-Select(n, s, f, v, p) M[0] = 0

for i = 1 to n

M[i] = 1 + M[p[i]]

return M[n]

Find-Solution(M, n) if n = 0

return {}

return {n} ∪ Find-Solution(p[n])

Activity-Selection Problem

Input: 𝑛 activities with 𝑠𝑖, 𝑓𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible Output: the maximum number of activities

(16)

## Greedy #2: Coin Changing

Textbook Exercise 16.1

(17)

𝑖

### • Cashier’s algorithm: at each iteration, add the coin with the largest value no more than the current total

Does this algorithm return the OPT?

(18)

### • Subproblems

• C(i): minimal number of coins for the total value 𝑖

• Goal: C(n)

Coin Changing Problem

Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛

(19)

### • Suppose OPT is an optimal solution to C(i) , there are 4 cases:

• Case 1: coin 1 in OPT

• OPT\coin1 is an optimal solution of C(i – v1)

• Case 2: coin 2 in OPT

• OPT\coin2 is an optimal solution of C(i – v2)

• Case 3: coin 3 in OPT

• OPT\coin3 is an optimal solution of C(i – v3)

• Case 4: coin 4 in OPT

• OPT\coin4 is an optimal solution of C(i – v4)

Coin Changing Problem

Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛

(20)

### • Proof via contradiction (use the case 10 ≤ 𝑖 < 50 for demo)

• Assume that there is no OPT including this greedy choice (choose 10)

→ all OPT use 1, 5, 50 to pay 𝑖

• 50 cannot be used

• #coins with value 5 < 2 → otherwise we can use a 10 to have a better output

• #coins with value 1 < 5 → otherwise we can use a 5 to have a better output

• We cannot pay 𝑖 with the constraints (at most 5 + 4 = 9) Coin Changing Problem

Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛

(21)

## Fractional Knapsack Problem

Textbook Exercise 16.2-2

(22)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(23)

𝑖

𝑖

𝑖

𝑖

### • Variants of knapsack problem

• 0-1 Knapsack Problem: 每項物品只能拿一個

• Unbounded Knapsack Problem: 每項物品可以拿多個

• Multidimensional Knapsack Problem: 背包空間有限

• Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

• Fractional Knapsack Problem: 物品可以只拿部分

(24)

𝑖

𝑖

𝑖

𝑖

𝑣𝑖

𝑤𝑖

𝑖

(25)

### • Subproblems

• F-KP(i, w): fractional knapsack problem within 𝑤 capacity for the first 𝑖 items

• Goal: F-KP(n, W)

Fractional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where we can take any fraction of items

(26)

### • Suppose OPT is an optimal solution to F-KP(i, w) , there are 2 cases:

• Case 1: full/partial item 𝑖 in OPT

• Remove 𝑤′ of item 𝑖 from OPT is an optimal solution of F-KP(i - 1, w – w)

• Case 2: item 𝑖 not in OPT

• OPT is an optimal solution of F-KP(i - 1, w) Fractional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where we can take any fraction of items

(27)

𝑣𝑖

𝑤𝑖

𝑖

𝑣𝑖 𝑤𝑖

### )

• Assume that there is no OPT including this greedy choice

• If 𝑊 ≤ 𝑤𝑗, we can replace all items in OPT with item 𝑗

• If 𝑊 > 𝑤𝑗, we can replace any item weighting 𝑤𝑗 in OPT with item 𝑗

• The total value must be equal or higher, because item 𝑗 has the highest 𝑣𝑖

𝑤𝑖

Do other knapsack problems have this property?

Fractional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where we can take any fraction of items

(28)

## Greedy #4: Breakpoint Selection

(29)

0

𝑛

0

𝑛

### • Greedy algorithm: go as far as you can before refueling

1 2 3 4 5

Ideally: stop when out of gas

Actually: may not be able to find the gas station when out of gas

1 2 3 4 5 6

(30)

### • Subproblems

• B(i): breakpoint selection problem from 𝑏𝑖 to 𝑏𝑛

• Goal: B(0)

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0→𝑏𝑛) that minimizes the number of stops

(31)

𝑗

𝑖

### ≤ 𝐶, there are 𝑗 − 𝑖 cases

• Case 1: stop at 𝑏𝑖+1

• OPT+{bi+1} is an optimal solution of B(i + 1)

• Case 2: stop at 𝑏𝑖+2

• OPT+{bi+2} is an optimal solution of B(i + 2) :

• Case 𝑗 − 𝑖: stop at 𝑏𝑗

• OPT+{bj} is an optimal solution of B(j) Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0→𝑏𝑛) that minimizes the number of stops

(32)

𝑗

### )

• Assume that there is no OPT including this greedy choice (after 𝑏𝑖 then stop at 𝑏𝑘, 𝑘 ≠ 𝑗)

• If 𝑘 > 𝑗, we cannot stop at 𝑏𝑘 due to out of gas

• If 𝑘 < 𝑗, we can replace the stop at 𝑏𝑘 with the stop at 𝑏𝑗

• The total value must be equal or higher, because we refuel later (𝑏𝑗 > 𝑏𝑘)

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0→𝑏𝑛) that minimizes the number of stops

(33)

### Pseudo Code

BP-Select(C, b)

Sort(b) s.t. b[0] < b[1] < … < b[n]

p = 0 S = {0}

for i = 1 to n - 1

if b[i + 1] – b[p] > C if i == p

return “no solution”

A = A ∪ {i}

p = i return A

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0→𝑏𝑛) that minimizes the number of stops

(34)

## Greedy #5: Huffman Codes

Textbook Chapter 16.3 – Huffman codes

(35)

### • Code (編碼) is a system of rules to convert information—such as a letter, word, sound, image, or gesture—into another, sometimes shortened or secret, form or representation for communication through a channel or storage in a medium.

input message

decoded message Encoder encoded Decoder

message

(36)

### • Goal

• Enable communication and storage

• Detect or correct errors introduced during transmission

• Compress data: lossy or lossless

Snoopy Encoder 536E6F6F7079 Decoder Snoopy

Encoder Decoder

(37)

(38)

### • Goal: encode each symbol using a unique binary code (w/o ambiguity)

• How to represent symbols?

• How to ensure decode(encode(x))=x?

• How to minimize the number of bits?

(39)

### • Goal: encode each symbol using a unique binary code (w/o ambiguity)

• How to represent symbols?

• How to ensure decode(encode(x))=x?

• How to minimize the number of bits?

0 1

0 1 0 1

10101101011010100101010010 T T C G G T T T G G G A T

find a binary tree

(40)

### • Fixed-length: use the same number of bits for encoding every symbol

• Ex. ASCII, Big5, UTF

### • The length of this sequence is

Fixed-length 000 001 010 011 100 101 Variable-length 0 101 100 111 1101 1100

E F

0 1

0 1

0 1

A B

0 1

C D

0 1

0

0 1

0 1

0 1

A

1

C D

1 0

E B

F 0

(41)

### • Goal: encode each symbol using an unique binary code (w/o ambiguity)

• How to represent symbols?

• How to ensure decode(encode(x))=x?

• How to minimize the number of bits?

use codes that are uniquely decodable

(42)

### Prefix Code

• Definition: a variable-length code where no codeword is a prefix of some other codeword

• Ambiguity: decode(1011100) can be ‘BF’ or ‘CDAA’

prefix codes are uniquely decodable

Symbol A B C D E F

Frequency (K) 45 13 12 16 9 5

Variable-length Prefix code 0 101 100 111 1101 1100 Not prefix code 0 101 10 111 1101 1100

(43)

### • Goal: encode each symbol using an unique binary code (w/o ambiguity)

• How to represent symbols?

• How to ensure decode(encode(x))=x?

• How to minimize the number of bits?

more frequent symbols should use shorter codewords

(44)

### Letter Frequency Distribution

shorter codewords longer codewords

(45)

### • Average bits per character

0 1

0

1

0 1

A:45

1

C:12 D:16

1 0

B:13

0 100

55

25 30

14 How to find the optimal prefix

code to minimize the cost?

(46)

1

2

𝑛

1

2

𝑛

(47)

### • Subproblem: merge two characters into a new one whose weight is their sum

• PC(i): prefix code problem for 𝑖 leaves

• Goal: PC(n)

### • Issues

• It is not the subproblem of the original problem

• The cost of two merged characters should be considered

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

PC(n) PC(n - 1)

(48)

0 1

0

1

0 1

A:45

1

C:12 D:16

1 0

E:9 B:13

F:5 0

100

55

25 30

14

0 1

1

0 1

A:45

C:12 D:16

1 0

B:13

0 100

55

25 30

EF:14

(49)

1…i-1

1…i-1

### , x, y})

z Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

(50)

z

x y

(51)

### • Optimal substructure: T’ is OPT if and only if T is OPT

T’ T

The difference is

(52)

### • Greedy choice: merge repeatedly until one tree left

• Select two trees 𝑥, 𝑦 with minimal frequency roots freq 𝑥 and freq 𝑦

• Merge into a single tree by adding root 𝑧 with the frequency freq 𝑥 + freq 𝑦

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

(53)

### Example

45

16 14

5 9

12 13

45

16 5 9 12 13

Initial set (store in a priority queue)

(54)

45

16 14

5 9

25

12 13

45

16 14

5 9

12 13

(55)

45

16

30

14

5 9

25

12 13

45

16 14

5 9

25

12 13

(56)

55 45

16

30

14

5 9

25

12 13

45

16

30

14

5 9

25

12 13

(57)

55 45

16

30

14

5 9

25

12 13

100

55 45

16

30

14

5 9

25

12 13

(58)

### • Greedy choice: merge two nodes with min weights repeatedly

• Assume that there is no OPT including this greedy choice

• 𝑥 and 𝑦 are two symbols with lowest frequencies

• 𝑎 and 𝑏 are siblings with largest depths

• WLOG, assume freq 𝑎 ≤ freq 𝑏 and freq 𝑥 ≤ freq 𝑦

→ freq 𝑥 ≤ freq 𝑎 and freq 𝑦 ≤ freq 𝑏

• Exchanging 𝑎 with 𝑥 and then 𝑏 with 𝑦 can make the tree equally or better

x

y

a b

OPT: T Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

(59)

### Step 3: Prove Greedy-Choice Property

x

y

a b

OPT: T

a

y

x b

T’

Because T is OPT, T’ must be another optimal solution.

(60)

### Step 3: Prove Greedy-Choice Property

x

y

a b

OPT: T

a

y

x b

T’

a

b

x y

T’’

Because T’ is OPT, T’’ must be another optimal solution.

(61)

### • Proof

• Use induction to prove: Huffman codes are optimal for 𝑛 symbols

• 𝑛 = 2, trivial

• For a set 𝑆 with 𝑛 + 1 symbols,

1. Based on the greedy choice property, two symbols with minimum frequencies are siblings in T

2. Construct T’ by replacing these two symbols 𝑥 and 𝑦 with 𝑧 s.t. 𝑆′ = (𝑆\{𝑥, 𝑦}) ∪ 𝑧 and freq 𝑧 = freq 𝑥 + freq 𝑦

3. Assume T’ is the optimal tree for 𝑛 symbols by inductive hypothesis

4. Based on the optimal substructure property, we know that when T’ is optimal, T is optimal too (case 𝑛 + 1 holds)

(62)

### Pseudo Code

Huffman(S) n = |S|

Q = Build-Priority-Queue(S) for i = 1 to n – 1

allocate a new node z

z.left = x = Extract-Min(Q) z.right = y = Extract-Min(Q) freq(z) = freq(x) + freq(y) Insert(Q, z)

Delete(Q, x) Delete(Q, y)

return Extract-Min(Q) // return the prefix tree Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

(63)

### • Huffman’s algorithm is sub-optimal when

• blending among symbols is allowed

• the probability distribution is unknown

• symbols are not independent

(64)

## To Be Continued…

(65)

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

• In the worst case, what is the growth of the function the optimal algorithm of the problem takes Design Step.

• “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution. • When to

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from

• Thinking: solve easiest case + combine smaller solutions into the original solution.. • Easy to find an

• makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.. • not always yield optimal solution; may end up at

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem. Prove that there is always an optimal solution to the

 This technique is solving a problem by reducing its instance to a smaller one, solving the latte r, and then extending the obtained solution to g et a solution to the

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem.. Prove that there is always an optimal solution to the

Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal

 “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution.  When to

Using the solution U of Riemann problem to obtain an e approximate solution for the solution U of balance laws..

In this work, for a locally optimal solution to the NLSDP (2), we prove that under Robinson’s constraint qualiﬁcation, the nonsingularity of Clarke’s Jacobian of the FB system

In this work, for a locally optimal solution to the nonlin- ear SOCP (4), under Robinson’s constraint qualification, we show that the strong second-order sufficient condition

 Definition: A problem exhibits  optimal substructure if an ..

 Definition: A problem exhibits optimal subst ructure if an optimal solution to the proble m contains within it optimal solutions to su bproblems..  怎麼尋找 optimal

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from

✓ Express the solution of the original problem in terms of optimal solutions for subproblems.. Construct an optimal solution from

▪ Approximation algorithms for optimization problems: the approximate solution is guaranteed to be close to the exact solution (i.e., the optimal value)..

 Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem. Prove that there is always an optimal solution to the

We will design a simple and effective iterative algorithm to implement “The parallel Tower of Hanoi problem”, and prove its moving is an optimal

proposed a greedy algorithm to utilize the Divide-and-Conquer technique to obtain near optimal scheduling while attempting to minimize the size of total communication messages

The proposed algorithms use the optimal-searching technique of genetic algorithm (GA) to get an efficient scheduling solution in grid computing environment and adapt to

The purpose of this research is to develop an optimum automatic formation model of the golf club head by using the optimal designing techniques to combine the techniques of the