• 沒有找到結果。

# Algorithm Design and Analysis Midterm Review

N/A
N/A
Protected

Share "Algorithm Design and Analysis Midterm Review"

Copied!
33
0
0

(1)

### Algorithm Design and Analysis

(2)

• Homework assignment

• HW2 due on 11/12 1pm

• HW2作業箱已擺在R217

• HW2作業解答會在死線當天晚上公布

• Midterm announcement

• Next week!!!

(3)

### Midterm!!!

• Date: 11/14 (Thursday)

• Time: 14:20-17:20 (3 hours)

• Location: R102 + R104 (check the seat assignment before entering the room)

• Content

• Recurrence and Asymptotic Analysis

• Divide and Conquer

• Dynamic Programming

• Greedy

• Based on slides, assignments, and some variations (practice via textbook exercises)

(4)

### Algorithm Design & Analysis Process

1) Formulate a problem 2) Develop an algorithm 3) Prove the correctness

4) Analyze running time/space requirement

Design Step

Analysis Step

(5)

### Algorithm Analysis

• Analysis Skills

• Induction

• Asymptotic analysis

• Problem instance

• Algorithm Complexity

• In the worst case, what is the growth of function an algorithm takes

• Problem Complexity

• In the worst case, what is the growth of the function the optimal algorithm of the

(6)

### Algorithm Design Strategy

• Do not focus on “specific algorithms”

• But “some strategies” to “design” algorithms

• First Skill: Divide-and-Conquer (各個擊破)

• Second Skill: Dynamic Programming (動態規劃)

• Third Skill: Greedy (貪婪法則)

(7)

## Divide-and-Conquer

(8)

### What is Divide-and-Conquer?

• Solve a problem recursively

• Apply three steps at each level of the recursion

1. Divide the problem into a number of subproblems that are smaller instances of the same problem (比較小的同樣問題)

2. Conquer the subproblems by solving them recursively If the subproblem sizes are small enough

• then solve the subproblems

• else recursively solve itself

3. Combine the solutions to the subproblems into the solution for the original problem

base case recursive case

1. Divide

2. Conquer

3. Combine

(9)

### How to Solve Recurrence Relations?

1. Substitution Method (取代法)

• Guess a bound and then prove by induction

2. Recursion-Tree Method (遞迴樹法)

• Expand the recurrence into a tree and sum up the cost

3. Master Method (套公式大法/大師法)

• Apply Master Theorem to a specific form of recurrences

(10)

### Master Theorem

compare 𝑓 𝑛 with 𝑛log𝑏 𝑎

divide a problem of size 𝑛 into 𝑎 subproblems, each of size 𝑛

𝑏 is solved in time 𝑇 𝑛

𝑏 recursively

The proof is in Ch. 4.6

(11)

### • Analyze the problem about

• Whether the problem with small inputs can be solved directly

• Whether subproblem solutions can be combined into the original solution

• Whether the overall complexity is better than naïve

### • If no, then

• Try another way for dividing

• Do not use D&C

(12)

### Pseudo-Polynomial Time

• Polynomial: polynomial in the length of the input (#bits for the input)

• Pseudo-polynomial: polynomial in the numeric value

• The time complexity of 0-1 knapsack problem is Θ 𝑛𝑊

• 𝑛: number of objects

• 𝑊: knapsack’s capacity (non-negative integer)

• polynomial in the numeric value

= pseudo-polynomial in input size

= exponential in the length of the input

• Note: the size of the representation of 𝑊 is log2 𝑊

= 2𝑚 = 𝑚

(13)

## Dynamic Programming

(14)

### What is Dynamic Programming?

• Dynamic programming, like the divide-and-conquer method, solves problems by combining the solutions to subproblems

• 用空間換取時間

• 讓走過的留下痕跡

• “Dynamic”: time-varying

• “Programming”: a tabular method

Dynamic Programming: planning over time

(15)

• Divide-and-Conquer

• partition the problem into

independent or disjoint subproblems

• repeatedly solving the common subsubproblems

→ more work than necessary

• Dynamic Programming

• partition the problem into dependent or overlapping subproblems

• avoid recomputation

✓ Top-down with memoization

✓ Bottom-up method

(16)

### Dynamic Programming Procedure

• Apply four steps

1. Characterize the structure of an optimal solution 2. Recursively define the value of an optimal solution

3. Compute the value of an optimal solution, typically in a bottom-up fashion 4. Construct an optimal solution from computed information

(17)

### When to Use DP?

• Whether subproblem solutions can combine into the original solution

• When subproblems are overlapping

• Whether the problem has optimal substructure

• Common for optimization problem

• Two ways to avoid recomputation

• Top-down with memoization

• Bottom-up method

• Complexity analysis

(18)

## Greedy Algorithms

(19)

### What is Greedy Algorithms?

• always makes the choice that looks best at the moment

• makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution

• not always yield optimal solution; may end up at local optimal

local maximal global maximal

local maximal

(20)

• Dynamic Programming

• has optimal substructure

• make an informed choice after getting optimal solutions to subproblems

• dependent or overlapping subproblems

• Greedy Algorithms

• has optimal substructure

• make a greedy choice before solving the subproblem

• no overlapping subproblems

✓ Each round selects only one subproblem

✓ The subproblem size decreases

Optimal Solution

Possible Case 1 Possible

Case 2

Possible Case k

max /min

Subproblem Solution Subproblem

Solution

Subproblem Solution

+

+

+

= Optimal

Solution

Greedy Choice

Subproblem Solution

= +

(21)

### Greedy Procedure

1. Cast the optimization problem as one in which we make a choice and remain one subproblem to solve

2. Demonstrate the optimal substructure

✓ Combining an optimal solution to the subproblem via greedy can arrive an optimal solution to the original problem

3. Prove that there is always an optimal solution to the original problem that makes the greedy choice

(22)

### Proof of Correctness Skills

• Optimal Substructure : an optimal solution to the problem contains within it optimal solutions to subproblems

• Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal solution

• Show that it exists an optimal solution that “contains” the greedy choice using exchange argument

• For any optimal solution OPT, the greedy choice 𝑔 has two cases

• 𝑔 is in OPT: done

• 𝑔 not in OPT: modify OPT into OPT’ s.t. OPT’ contains 𝑔 and is at least as good as OPT

OPT OPT’

𝑔

If OPT’ is better than OPT, the property is proved by contradiction

If OPT’ is as good as OPT, then we showed that there exists an optimal solution containing 𝑔 by construction

(23)

### When to Use Greedy?

• Whether the problem has optimal substructure

• Whether we can make a greedy choice and remain only one subproblem

• Common for optimization problem

Optimal Solution

Greedy Choice

Subproblem Solution

= +

(24)

## Exercises

(25)

• True or False: To prove the correctness of a greedy algorithm, we must prove that every optimal solution contains our greedy choice.

• Given the following recurrence relation, provide a valid traversal order to fill the DP table or justify why no valid traversal exists.

(26)

### Matrix-Chain Multiplication

• Input: a sequence of integers 𝑙0, 𝑙1, … , 𝑙𝑛

• 𝑙𝑖−1 is the number of rows of matrix 𝐴𝑖

• 𝑙𝑖 is the number of columns of matrix 𝐴𝑖

• Output: an order of performing 𝑛 − 1 matrix multiplications in the maximum number of operations to obtain the product of 𝐴1𝐴2 … 𝐴𝑛

𝐴1 𝐴2 𝐴3 𝐴4 …… 𝐴𝑛

𝐴1.cols=𝐴2.rows

𝐴𝑖𝐴𝑖+1 … 𝐴𝑘 𝐴𝑘+1𝐴𝑘+2 … 𝐴𝑗 𝑖 ≤ 𝑘 < 𝑗

Q: Does optimal substructure still hold?

(27)

### Painting

• Put stickers in a single row on each tube to indicate its color.

• There are 𝑘 types of stickers.

• Tubes with the same color should have the same sticker pattern and should be prefix free.

• Minimize the total number of stickers put on all tubes

Color red pink orange yellow green blue purple black

#Tubes 25 15 12 19 7 12 8 2

(28)

### 3-arry Huffman Coding

• The total length is

Why?

Color red pink orange yellow green blue purple black

#Tubes 25 15 12 19 7 12 8 2

(29)

### T/F Question

• Given a file containing a sequence of 8-bit characters (256 characters), if the maximum character frequency is less than k of the minimum character

frequency in the file, then a binary Human code is always worse than or equal to an 8-bit fixed length code (in terms of the length of the encoded file).

• What is the minimal value of k?

https://stackoverflow.com/questions/8960698/huffman-coding-prove-on-a-8-bit- sequence

(30)

(31)

(32)

(33)

## Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Greedy-Choice Property : making locally optimal (greedy) choices leads to a globally optimal

 “Greedy”: always makes the choice that looks best at the moment in the hope that this choice will lead to a globally optimal solution.  When to

Using the solution U of Riemann problem to obtain an e approximate solution for the solution U of balance laws..

In this work, for a locally optimal solution to the NLSDP (2), we prove that under Robinson’s constraint qualiﬁcation, the nonsingularity of Clarke’s Jacobian of the FB system

In this work, for a locally optimal solution to the nonlin- ear SOCP (4), under Robinson’s constraint qualification, we show that the strong second-order sufficient condition

 Definition: A problem exhibits  optimal substructure if an ..

 Definition: A problem exhibits optimal subst ructure if an optimal solution to the proble m contains within it optimal solutions to su bproblems..  怎麼尋找 optimal

✓ Express the solution of the original problem in terms of optimal solutions for subproblems. Construct an optimal solution from