• 沒有找到結果。

# Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

N/A
N/A
Protected

Share "Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai"

Copied!
66
0
0

(1)

Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

(2)

### Mini-HW 6 Released

Due on 11/09 (Thu) 17:20

### Homework 2

Due on 11/09 (Thur) 17:20

### Midterm

Time: 11/16 (Thur) 14:20-17:20

Format: close book

Location: R103 (please check the assigned seat before entering the room)

2

Frequently check the website for the updated information!

(3)

3

(4)

4

(5)

### Third Skill: Greedy (貪婪法則)

5

(6)

Textbook Chapter 16 – Greedy Algorithms

Textbook Chapter 16.2 – Elements of the greedy strategy

## 6

(7)

### not always yield optimal solution; may end up at local optimal

7

Greedy: move towards max gradient and hope it is global maximum local maximal

global maximal

local maximal

(8)

### Dynamic Programming

has optimal substructure

make an informed choice after getting optimal solutions to subproblems

dependent or overlapping subproblems

### Greedy Algorithms

has optimal substructure

make a greedy choice before solving the subproblem

no overlapping subproblems

Each round selects only one subproblem

The subproblem size decreases

8

Optimal Solution

Possible Case 1 Possible

Case 2

Possible Case k

max /min

Subproblem Solution Subproblem

Solution

Subproblem Solution

+ +

+

=

Optimal Solution

Greedy Choice

Subproblem Solution

= +

(9)

9

(10)

10

(11)

### Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal solution

Show that it exists an optimal solution that “contains” the greedy choice using exchange argument

For any optimal solution OPT, the greedy choice 𝑔 has two cases

𝑔 is in OPT: done

𝑔 not in OPT: modify OPT into OPT’ s.t. OPT’ contains 𝑔 and is at least as good as OPT

11

OPT OPT’

𝑔

✓ If OPT’ is better than OPT, the property is proved by contradiction

✓ If OPT’ is as good as OPT, then we showed that there exists an optimal solution containing 𝑔 by construction

(12)

Textbook Chapter 16.1 – An activity-selection problem

## 12

(13)

𝑖

𝑖

1

2

𝑛

1

2

𝑛

1

2

### < ⋯ < 𝑓

𝑛

time 13

1 2 3 4 5 6

activity index

1 2 3 4 5 6 7 8 9

(14)

### Subproblems

WIS(i): weighted interval scheduling for the first 𝑖 jobs

Goal: WIS(n)

Dynamic programming algorithm

14

i 0 1 2 3 4 5 n

M[i]

Weighted Interval Scheduling Problem

Input: 𝑛 jobs with 𝑠𝑖, 𝑓𝑖, 𝑣𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. jobs 𝑖 and 𝑗 are compatible Output: the maximum total value obtainable from compatible

Set 𝑣𝑖 = 1 for all 𝑖 to formulate it into the activity-selection problem

(15)

Dynamic programming

Greedy algorithm

15

Activity-Selection Problem

Input: 𝑛 activities with 𝑠𝑖, 𝑓𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible Output: the maximum number of activities

select the 𝑖-th activity

Why does the 𝑖-th activity must appear in an OPT?

(16)

### Proof

Assume there is an OPT solution for the first 𝑖 − 1 activities (𝑀𝑖−1)

𝐴𝑗 is the last activity in the OPT solution 

Replacing 𝐴𝑗 with 𝐴𝑖 does not make the OPT worse

time 16

1 2 : i i - 1

:

activity index

1 2 3 4 5 6 7 8 9

(17)

17

Act-Select(n, s, f, v, p) M[0] = 0

for i = 1 to n if p[i] >= 0

M[i] = 1 + M[p[i]]

return M[n]

Find-Solution(M, n) if n = 0

return {}

return {n} ∪ Find-Solution(p[n])

Activity-Selection Problem

Input: 𝑛 activities with 𝑠𝑖, 𝑓𝑖 , 𝑝(𝑗) = largest index 𝑖 < 𝑗 s.t. 𝑖 and 𝑗 are compatible Output: the maximum number of activities

Select the last compatible one (←) = Select the first compatible one (→)

(18)

Textbook Exercise 16.1

## 18

(19)

𝑖

### Cashier’s algorithm: at each iteration, add the coin with the largest value no more than the current total

19

Does this algorithm return the OPT?

(20)

### Subproblems

C(i): minimal number of coins for the total value 𝑖

Goal: C(n)

20

Coin Changing Problem

Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛

(21)

### Suppose OPT is an optimal solution to C(i), there are 4 cases:

Case 1: coin 1 in OPT

OPT\coin1 is an optimal solution of C(i – v1)

Case 2: coin 2 in OPT

OPT\coin2 is an optimal solution of C(i – v2)

Case 3: coin 3 in OPT

OPT\coin3 is an optimal solution of C(i – v3)

Case 4: coin 4 in OPT

OPT\coin4 is an optimal solution of C(i – v4)

21

Coin Changing Problem

Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛

(22)

### Proof via contradiction (use the case 10 ≤ 𝑖 < 50 for demo)

Assume that there is no OPT including this greedy choice (choose 10)

 all OPT use 1, 5, 50 to pay 𝑖

50 cannot be used

#coins with value 5 < 2  otherwise we can use a 10 to have a better output

#coins with value 1 < 5  otherwise we can use a 5 to have a better output

We cannot pay 𝑖 with the constraints (at most 5 + 4 = 9)

22

Coin Changing Problem

Input: 𝑛 dollars and unlimited coins with values 𝑣𝑖 (1, 5, 10, 50) Output: the minimum number of coins with the total value 𝑛

(23)

Textbook Exercise 16.2-2

## 23

(24)

𝑖

𝑖

𝑖

𝑖

### Variants of knapsack problem

0-1 Knapsack Problem: 每項物品只能拿一個

Unbounded Knapsack Problem: 每項物品可以拿多個

Multidimensional Knapsack Problem: 背包空間有限

Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

Fractional Knapsack Problem: 物品可以只拿部分

24

(25)

𝑖

𝑖

𝑖

𝑖

### Variants of knapsack problem

0-1 Knapsack Problem: 每項物品只能拿一個

Unbounded Knapsack Problem: 每項物品可以拿多個

Multidimensional Knapsack Problem: 背包空間有限

Multiple-Choice Knapsack Problem: 每一類物品最多拿一個

Fractional Knapsack Problem: 物品可以只拿部分

25

(26)

𝑖

𝑖

𝑖

𝑖

𝑣𝑖

𝑤𝑖

𝑖

26

(27)

### Subproblems

F-KP(i, w): fractional knapsack problem within 𝑤 capacity for the first 𝑖 items

Goal: F-KP(n, W)

27

Fractional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where we can take any fraction of items

(28)

### Suppose OPT is an optimal solution to F-KP(i, w), there are 2 cases:

Case 1: full/partial item 𝑖 in OPT

Remove 𝑤′ of item 𝑖 from OPT is an optimal solution of F-KP(i - 1, w – w)

Case 2: item 𝑖 not in OPT

OPT is an optimal solution of F-KP(i - 1, w)

28

Fractional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where we can take any fraction of items

(29)

𝑣𝑖

𝑤𝑖

𝑖

𝑣𝑖 𝑤𝑖

### )

Assume that there is no OPT including this greedy choice

If 𝑊 ≤ 𝑤𝑗, we can replace all items in OPT with item 𝑗

If 𝑊 > 𝑤𝑗, we can replace any item weighting 𝑤𝑗 in OPT with item 𝑗

The total value must be equal or higher, because item 𝑗 has the highest 𝑣𝑖

𝑤𝑖

29

Fractional Knapsack Problem

Input: 𝑛 items where 𝑖-th item has value 𝑣𝑖 and weighs 𝑤𝑖

Output: the max value within 𝑊 capacity, where we can take any fraction of items

Do other knapsack problems have this property?

(30)

## 30

(31)

0

𝑛

0

𝑛

### Greedy algorithm: go as far as you can before refueling

31

1 2 3 4 5

Ideally: stop when out of gas

Actually: may not be able to find the gas station when out of gas

1 2 3 4 5 6

(32)

### Subproblems

B(i): breakpoint selection problem from 𝑏𝑖 to 𝑏𝑛

Goal: B(0)

32

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0𝑏𝑛) that minimizes the number of stops

(33)

𝑗

𝑖

### ≤ 𝐶, there are 𝑗 − 𝑖 cases

Case 1: stop at 𝑏𝑖+1

OPT+{bi+1} is an optimal solution of B(i + 1)

Case 2: stop at 𝑏𝑖+2

OPT+{bi+2} is an optimal solution of B(i + 2) :

Case 𝑗 − 𝑖: stop at 𝑏𝑗

OPT+{bj} is an optimal solution of B(j)

33

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0𝑏𝑛) that minimizes the number of stops

(34)

𝑗

### )

Assume that there is no OPT including this greedy choice (after 𝑏𝑖 then stop at 𝑏𝑘, 𝑘 ≠ 𝑗)

If 𝑘 > 𝑗, we cannot stop at 𝑏𝑘 due to out of gas

If 𝑘 < 𝑗, we can replace the stop at 𝑏𝑘 with the stop at 𝑏𝑗

The total value must be equal or higher, because we refuel later (𝑏𝑗 > 𝑏𝑘)

34

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0𝑏𝑛) that minimizes the number of stops

(35)

35

Breakpoint Selection Problem

Input: 𝑛 + 1 breakpoints 𝑏0, … , 𝑏𝑛; gas storage is 𝐶

Output: a refueling schedule (𝑏0𝑏𝑛) that minimizes the number of stops

BP-Select(C, b)

Sort(b) s.t. b[0] < b[1] < … < b[n]

p = 0 S = {0}

for i = 1 to n - 1

if b[i + 1] – b[p] > C if i == p

return “no solution”

A = A ∪ {i}

p = i return A

(36)

Textbook Chapter 16.3 – Huffman codes

## 36

(37)

37

Encoder

Decoder

(38)

38

### Snoopy

Encoder 536E6F6F7079 Decoder

Encoder Decoder

(39)

39

### Goal: encode each symbol using an unique binary code (w/o ambiguity)

How to represent symbols?

How to ensure decode(encode(x))=x?

How to minimize the number of bits?

(40)

40

### Goal: encode each symbol using an unique binary code (w/o ambiguity)

How to represent symbols?

How to ensure decode(encode(x))=x?

How to minimize the number of bits?

A G T C

0 1

0 1 0 1

10101101011010100101010010 T T C G G T T T G G G A T find a binary tree

(41)

### Fixed-length: use the same number of bits for encoding every symbol

Ex. ASCII, Big5, UTF

### The length of this sequence is

41

Symbol A B C D E F

Frequency (K) 45 13 12 16 9 5

Fixed-length 000 001 010 011 100 101

Variable-length 0 101 100 111 1101 1100

E F

0 1

0 1

0 1

A B

0 1

C D

0 1

0

0 1

0 1

0 1

A

1

C D

1 0

E B

F 0

(42)

42

### Goal: encode each symbol using an unique binary code (w/o ambiguity)

How to represent symbols?

How to ensure decode(encode(x))=x?

How to minimize the number of bits?

use codes that are uniquely decodable

(43)

43

### prefix codes are uniquely decodable

Symbol A B C D E F

Frequency (K) 45 13 12 16 9 5

Variable-length Prefix code 0 101 100 111 1101 1100

Not prefix code 0 101 10 111 1101 1100

(44)

44

### Goal: encode each symbol using an unique binary code (w/o ambiguity)

How to represent symbols?

How to ensure decode(encode(x))=x?

How to minimize the number of bits?

more frequent symbols should use shorter codewords

(45)

45

shorter codewords longer codewords

(46)

46

### Average bits per character

0 1

0

1

0 1

A:45

1

C:12 D:16

1 0

E:9 B:13

F:5 0

100

55

25 30

How to find the optimal prefix 14 code to minimize the cost?

(47)

1

2

𝑛

1

2

𝑛

47

(48)

### Subproblem: merge two characters into a new one whose weight is their sum

PC(i): prefix code problem for 𝑖 leaves

Goal: PC(n)

### Issues

It is not the subproblem of the original problem

The cost of two merged characters should be considered

48

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

PC(n)  PC(n - 1)

(49)

49

0 1

0

1

0 1

A:45

1

C:12 D:16

1 0

E:9 B:13

F:5 0

100

55

25 30

14

0 1

1

0 1

A:45

C:12 D:16

1 0

B:13

0 100

55

25 30

EF:14

(50)

1…i-1

### , z})

50

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

z

x y

1…i-1

(51)

51

z

x y

(52)

### Optimal substructure: T is OPT if and only if T’ is OPT

52

T T’

The difference is

(53)

### Greedy choice: merge repeatedly until one tree left

Select two trees 𝑥, 𝑦 with minimal frequency roots freq 𝑥 and freq 𝑦

Merge into a single tree by adding root 𝑧 with the frequency freq 𝑥 + freq 𝑦

53

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

(54)

54

45

16 14

5 9

12 13

45

16 5 9 12 13

Initial set (store in a priority queue)

(55)

55

45

16 14

5 9

25

12 13

45

16 14

5 9

12 13

(56)

56

45

16

30

14

5 9

25

12 13

45

16 14

5 9

25

12 13

(57)

57

55 45

16

30

14

5 9

25

12 13

45

16

30

14

5 9

25

12 13

(58)

58

55 45

16

30

14

5 9

25

12 13

100

55 45

16

30

14

5 9

25

12 13

(59)

### Greedy choice: merge two nodes with min weights repeatedly

Assume that there is no OPT including this greedy choice

𝑥 and 𝑦 are two symbols with lowest frequencies

𝑎 and 𝑏 are siblings with largest depths

WLOG, assume freq 𝑎 ≤ freq 𝑏 and freq 𝑥 ≤ freq 𝑦

 freq 𝑥 ≤ freq 𝑎 and freq 𝑦 ≤ freq 𝑏

Exchanging 𝑎 with 𝑥 and then 𝑏 with 𝑦 can make the tree equally or better

59

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

x

y

a b

OPT: T

(60)

60

x

y

a b

OPT: T

a

y

x b

T’

(61)

61

x

y

a b

OPT: T

a

y

x b

T’

a

b

x y

T’’

(62)

### Use induction to prove: Huffman codes are optimal for 𝑛 symbols

𝑛 = 2, trivial

For a set 𝑆 with 𝑛 + 1 symbols,

1. Based on the greedy choice property, two symbols with minimum frequencies are siblings in T

2. Construct T’ by replacing these two symbols 𝑥 and 𝑦 with 𝑧 s.t. 𝑆′ = (𝑆\{𝑥, 𝑦}) ∪ 𝑧 and freq 𝑧 = freq 𝑥 + freq 𝑦

3. Assume T’ is the optimal tree for 𝑛 symbols by inductive hypothesis

4. Based on the optimal substructure property, we know that when T’ is optimal, T is optimal too (case 𝑛 + 1 holds)

62

This induction proof framework can be applied to prove its optimality using the optimal substructure and the greedy choice property.

(63)

63

Huffman(S) n = |S|

Q = Build-Priority-Queue(S) for i = 1 to n – 1

allocate a new node z

z.left = x = Extract-Min(Q) z.right = y = Extract-Min(Q) freq(z) = freq(x) + freq(y) Insert(Q, z)

return Extract-Min(Q) // return the prefix tree

Prefix Code Problem

Input: 𝑛 positive integers 𝑤1, 𝑤2, … , 𝑤𝑛 indicating word frequency Output: a binary tree of 𝑛 leaves with minimal cost

(64)

64

(65)

(66)

## 66

Important announcement will be sent to @ntu.edu.tw mailbox

& post to the course website

• “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution. • When to

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from

• makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.. • not always yield optimal solution; may end up at

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem. Prove that there is always an optimal solution to the

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem.. Prove that there is always an optimal solution to the

Using

Calculate the amortized cost of each operation based on the potential function. Calculate total amortized cost based on

vertices’ edges, in this shortest path, the left edge must be relaxed before the right edge.  One phase of improvement

 Combine: find closet pair with one point in each region, and return the best of three

▪ Step 2: Run DFS on the transpose

Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal

 From a source vertex, systematically follow the edges of a graph to visit all reachable vertices of the graph.  Useful to discover the structure of

 “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution.  When to

 Definition: A problem exhibits  optimal substructure if an ..

 Definition: A problem exhibits optimal subst ructure if an optimal solution to the proble m contains within it optimal solutions to su bproblems..  怎麼尋找 optimal

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from

✓ Express the solution of the original problem in terms of optimal solutions for subproblems.. Construct an optimal solution from

 If SAT can be solved in deterministic polynomial time, then so can any NP problems  SAT ∈ NP-hard..  If A is an NP-hard problem and B can be reduced from A, then B is an

▪ Approximation algorithms for optimization problems: the approximate solution is guaranteed to be close to the exact solution (i.e., the optimal value)..

 Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem. Prove that there is always an optimal solution to the

▪ Can we decide whether a problem is “too hard to solve” before investing our time in solving it.. ▪ Idea: decide which complexity classes the problem belongs to

We will design a simple and effective iterative algorithm to implement “The parallel Tower of Hanoi problem”, and prove its moving is an optimal

The proposed algorithms use the optimal-searching technique of genetic algorithm (GA) to get an efficient scheduling solution in grid computing environment and adapt to