• 沒有找到結果。

# Algorithm Design and Analysis Amortized Analysis

N/A
N/A
Protected

Share "Algorithm Design and Analysis Amortized Analysis"

Copied!
33
0
0

(1)

### Algorithm Design and AnalysisAmortized Analysis

Yun-Nung (Vivian) Chen

(2)

### Outline

• Amortized analysis

• #1: Stack Operations

• Aggregate method

• Accounting method

• Potential method

• #2: Binary Counter

• Aggregate method

• Accounting method

• Potential method

(3)

### • Design Strategy

• Divide-and-Conquer

• Dynamic Programming

• Greedy Algorithms

• Graph Algorithms

### • Analysis

• Amortized Analysis

(4)

### Amortized Analysis

Textbook Chapter 17 – Amortized Analysis

(5)

### • A data structure comes with operations that organize the stored data

• Different operations may have different costs

• The same operation may have different costs

stack

PUSH POP

MULTIPOP

cost

operations

(6)

### Worst Case Time Complexity

stack

PUSH POP

MULTIPOP

cost

operations worst-case

Cost of stack operations PUSH(S, x) = O(1)

POP(S) = O(1)

MULTIPOP(S, k) = O(min(|S|, k))

(7)

### Worst Case Time Complexity

• n-th operation takes MULTIPOP(S, n) = O(n) time in the worst case

• n operations take O(n2) time

Stack Operations

Suppose that we apply a sequence of n operations on a data structure. What is the time complexity of the procedure?

Can this be an over-estimate?

What if only a few operations take O(n) time and the rest of them take O(1) time?

The worst-case bound is not tight because this

expensive Multipop operation cannot occur so frequently!

(8)

### Amortized Analysis

• Goal: obtain an accurate worst-case bound in executing a sequence of operations on a given data structure

• An upper bound for any sequence of n operations

• Comparison: types of running-time analysis

Type Description

Worst case Running time guarantee for any input of size n

Average case Expected running time for a random input of size n Probabilistic Expected running time of a randomized algorithm

Amortized Worst-case running time for a sequence of n operations

(9)

## Aggregate method (聚集法)

• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations

• The average cost per operation is then 𝑇(𝑛)/𝑛

• All operations have the same amortized cost

Accounting method (記帳法)

• Each operation is assigned an amortized cost (may differ from the actual cost)

• Each object of the data structure is associated with a credit

• Need to ensure that every object has sufficient credit at any time

Potential method (位能法)

• Similar to accounting method; each operation is assigned an amortized cost

• The data structure as a whole maintains a credit (i.e., potential)

• Need to ensure that the potential level is nonnegative at any time

(10)

### Stack Operations

Textbook Chapter 17.1 – Aggregate analysis

Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method

(11)

### Stack Operations

• Implementation with an array or a linked list

Operation Type Cost

PUSH(S, x): inset an element x into S POP(S): pop the top element from S

MULTIPOP(S, k): pop top k elements from S at once

stack

PUSH POP

MULTIPOP MULTIPOP(S, k)

while not STACK-EMPTY(S) and k > 0 POP(S)

k = k - 1

Stack Operations

Suppose that we apply a sequence of n operations on a data structure. What is the time complexity of the procedure?

(12)

### Aggregate Method (聚集法)

• Approach:

1. Determine an upper bound 𝑇(𝑛) on the cost of any sequence of 𝑛 operations 2. Calculate the amortized cost per operation as 𝑇(𝑛)/𝑛

3. All operations have the same amortized cost

cost

operations 𝑇(𝑛) Amortized cost of each op =

… …

(13)

### Aggregate Method for Stack

• The number of each operation type

• These npop + nmultipop operations together take at most

• Total cost for n operations:

• Amortized cost per operation:

Operation Type #Operations

PUSH(S, x): inset an element x into S npush POP(S): pop the top element from S npop MULTIPOP(S, k): pop top k elements from S at once nmultipop

n

Key idea: #pop elements ≤ #push operations/elements

(14)

### Another Thinking

• Once the push operation is taken, we prepare the additional cost for the future usage of multipop

Key idea: #pop elements ≤ #push operations/elements

(15)

### Accounting Method (記帳法)

• Idea: save credits from the operations that take less cost for future use of

operations that take more cost (針對使用花費較低的operations時先存錢未 雨綢繆, 供未來花費較高的operations使用)

• Approach:

1. Each operation is assigned a valid amortized cost

• If amortized cost > actual cost, the difference becomes credit (存)

• Credit is deposited in an object of the data structure

• If amortized cost < actual cost, then withdraw (提) stored credits

2. Validity check: ensure that every object has sufficient credit for any sequence of n operations

3. Calculate total amortized cost based on individual ones

(16)

### Accounting Method (記帳法)

• Validity check: ensure that every object has sufficient credit for any times of n operations (不能有赤字)

• ci: the actual cost of the i-th operation

• ĉi: the amortized cost of the i-th operation

→ For all sequences of n operations, we require

Aggregate Method

Each type of operations have its actual cost

Compute amortized cost using T(n)

Accounting Method

Each type of operations can have a different amortized cost

Assign valid amortized costs first and then compute T(n)

(17)

### Accounting Method for Stack

1. Assign the amortized cost

2. Show that for each object s.t.

• PUSH: the pushed element is deposited \$1 credit

• POP and MULTIPOP: use the credit stored with the popped element

• There is always enough credit to pay for each operation

3. Each amortized cost is O(1) → total amortized cost is O(n)

Operation Type Actual Cost Amortized Cost

PUSH(S, x) 1 2

POP(S) 1 0

MULTIPOP(S, k) min(|S|, k) 0

(18)

### Potential Method (位能法)

• Idea: represent the prepaid work as “potential,” which can be released to pay for future operations (the potential is associated with the whole data

structure rather than specific objects)

• Approach:

1. Select a potential function that takes the current data structure state as input and outputs a “potential level”

2. Validity check: ensure that the potential level is nonnegative

3. Calculate the amortized cost of each operation based on the potential function 4. Calculate total amortized cost based on individual ones

Accounting Method

Each object within the data structure has its credit

Potential Method

The data structure has credits

(19)

### Potential Method (位能法)

• Potential function Φ maps any state of the data structure to a real number

• D0: the initial state of data structure

• Di: the state of data structure after i-th operation

• ci: the actual cost of i-th operation

• ĉi: the amortized cost of i-th operation, defined as

(20)

### Potential Method (位能法)

• Total amortized cost

• To obtain an upper bound on the actual cost

• Define a potential function such that

• Usually we set

(21)

### Potential Method for Stack

1. Define Φ 𝐷𝑖 to be the number of elements in the stack after the i-th operation

2. Validity check:

• The stack is initially empty →

• The number of elements in the stack is always ≥ 0 →

3. Compute amortized cost of each operation:

• PUSH(S, X):

• POP(S):

• MULTIPOP(S, k):

4. All operations have O(1) amortized cost → total amortized cost is O(n)

Practice: justify why it is zero

ci: the actual cost of i-th operation ĉi: the amortized cost of i-th operation

(22)

## Fibonacci Heap

(23)

### Prim’s Time Complexity

• Fibonacci heap (Textbook Ch. 19)

• BUILD-MIN-HEAP:

• EXTRACT-MIN: (amortized)

• DECREASE-KEY: (amortized)

• Total complexity:

MST-PRIM(G, w, r) // w = weights, r = root for u in G.V

u.key = ∞ u.π = NIL r.key = 0 Q = G.V

while Q ≠ empty

u = EXTRACT-MIN(Q) for v in G.adj[u]

if v ∈ Q and w(u, v) < v.key v.π = u

v.key = w(u, v) // DECREASE-KEY

(24)

### Dijkstra’s Time Complexity

• Fabonacci heap (Textbook Ch. 19)

• BUILD-MIN-HEAP:

• EXTRACT-MIN: (amortized)

• DECREASE-KEY: (amortized)

• Total complexity:

DIJKSTRA(G, w, s)

INITIALIZATION(G, s) S = empty

Q = G.v // INSERT while Q ≠ empty

u = EXTRACT-MIN(Q) S = S∪{u}

RELAX(u, v, w)

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

(25)

### Binary Counter

Textbook Chapter 17.1 – Aggregate analysis

Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method

(26)

### Binary Counter

• Implementation with a k-bit array

• Each operation takes O(log n) time in the worst case

• n operations take O(n log n) time

INCREMENT(A) i = 0

while i < A.length and A[i] == 1 A[i] = 0

i = i + 1

if i < A.length A[i] = 1

Binary Counter

Suppose that a counter is initially zero. We increment the counter n times. How many bits are altered throughout the process?

0 1 10 11 100 101 110 111 1000

1001 1010 1011 1100 1101 1110 1111 10000 increment

(27)

### Aggregate Method for Binary Counter

Counter

Value A A A A Total Cost of First n Operations

0 0 0 0 0 0

1 0 0 0 1 1

2 0 0 1 0 3

3 0 0 1 1 4

4 0 1 0 0 7

5 0 1 0 1 8

6 0 1 1 0 10

7 0 1 1 1 11

8 1 0 0 0 15

flip every increment flip every 2 increments

flip every 4 increments flip every 8 increments

(28)

### Aggregate Method for Binary Counter

• Total #bits flipping in n increment operations:

• Total cost of the sequence:

• Amortized cost per operation:

(29)

### Accounting Method for Binary Counter

1. Assign the amortized cost

2. Validity check:

• Each bit 0 to bit 1, we save additional \$1 in the bit 1

• When bit 1 becomes to bit 0, we spend the saved cost

3. Each increment

• Change many 1s to 0s → free

• Change exactly a 0 to 1 → O(1)

• Each amortized cost is O(1) → total amortized cost is O(n)

Operation Actual Cost Amortized Cost

bit 0 → bit 1 1 2 (存\$1到bit 1)

bit 1 → bit 0 1 0 (用掉存在bit 1裡面的\$1)

increment #flipped bits 2 for setting a bit to 1

(30)

### Accounting Method for Binary Counter

Counter

Value A A A A Total Cost of First n

Operations

0 0 0 0 0 0

1 0 0 0 1 1

2 0 0 1 0 3

3 0 0 1 1 4

4 0 1 0 0 7

5 0 1 0 1 8

6 0 1 1 0 10

7 0 1 1 1 11

8 1 0 0 0 15

(31)

### Potential Method for Binary Counter

1. Define Φ 𝐷𝑖 to be the number of 1s in the counter after the i-th operation

2. Validity check:

• The counter is initially zero →

• The number of 1’s cannot be negative →

3. Compute amortized cost of each INCREMENT:

• Let LSB0(i) be the number of continuous 1s in the suffix

• For example, LSB0(01011011) = 2, and LSB0(01011111) = 5

4. All operations have O(1) amortized cost → total amortized cost is O(n)

ci: the actual cost of i-th operation ĉi: the amortized cost of i-th operation

(32)

### Concluding Remarks

Aggregate method (聚集法)

• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations

• The average cost per operation is then 𝑇(𝑛)/𝑛

• All operations have the same amortized cost

Accounting method (記帳法)

• Each operation is assigned an amortized cost (may differ from the actual cost)

• Each object of the data structure is associated with a credit

• Need to ensure that every object has sufficient credit at any time

Potential method (位能法)

• Similar to accounting method; each operation is assigned an amortized cost

• The data structure as a whole maintains a credit (i.e., potential)

• Need to ensure that the potential level is nonnegative at any time

(33)

## Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

a smooth cost function, so that the cost of an instance should be similar with its neighbors’.On the basis of the extended idea, we propose the cost-sensitive tree sampling

• In the worst case, what is the growth of the function the optimal algorithm of the problem takes Design Step.

• Thinking: solve easiest case + combine smaller solutions into the original solution.. • Easy to find an

• makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution.. • not always yield optimal solution; may end up at

• Tree lifetime: When the first node is dead in tree T, the rounds number of the node surviving is the lifetime of the tree. The residual energy of node is denoted as E)), where

Calculate the amortized cost of each operation based on the potential function. Calculate total amortized cost based on

In particular, we present a linear-time algorithm for the k-tuple total domination problem for graphs in which each block is a clique, a cycle or a complete bipartite graph,

Corpus-based information ― The grammar presentations are based on a careful analysis of the billion-word Cambridge English Corpus, so students and teachers can be

Lin, A smoothing Newton method based on the generalized Fischer-Burmeister function for MCPs, Nonlinear Analysis: Theory, Methods and Applications, 72(2010), 3739-3758..

allocate new-table with 2*T.size slots insert all items in T.table into new- table.

 “More Joel on Software : Further Thoughts on Diverse and Occasionally Related Matters That Will Prove of Interest to Software Developers, Designers, and Managers, and to Those

Financial Analysis (i) Calculate ratios and comment on a company’s profitability, liquidity, solvency, management efficiency and return on investment: mark-up, inventory

Based on historical documents and archeological evidence, this thesis provides an analysis of, raises some worth-noting questions on, the development of Western Qin Buddhism

CAST: Using neural networks to improve trading systems based on technical analysis by means of the RSI financial indicator. Performance of technical analysis in growth and small

CAST: Using neural networks to improve trading systems based on technical analysis by means of the RSI financial indicator. Performance of technical analysis in growth and small

• If we want analysis with amortized costs to show that in the worst cast the average cost per operation is small, the total amortized cost of a sequence of operations must be

In this chapter, the results for each research question based on the data analysis were presented and discussed, including (a) the selection criteria on evaluating

Based on the tourism and recreational resources and lodging industry in Taiwan, this paper conducts the correlation analysis on spatial distribution of Taiwan

Soille, “Watershed in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations,” IEEE Transactions on Pattern Analysis and Machine Intelligence,

Based on the insertion of redundant wires and the analysis of the clock skew in a clock tree, an efficient OPE-aware algorithm is proposed to repair the zero-skew

Thermo analysis technology is based on the charts and patterns of measurements from the product’s thermogravimetric analysis (TGA) and differential thermal analyzis (DTA), and

The process of optimization is as shown below: (1) the design is generated based on uniform random number, (2) the structural analysis is conducted by structural analysis

Based on the analysis of experiential benefits, there was significant difference on “psychological and social” between visitors’ background, like place of residence and