Slides credited from Hsueh-I Lu & Hsu-Chun Hsiao

### ▪ Amortized analysis

### ▪ #1: Stack Operations

### ▪ Aggregate method

### ▪ Accounting method

### ▪ Potential method

### ▪ #2: Binary Counter

### ▪ Aggregate method

### ▪ Accounting method

### ▪ Potential method

**2**

### ▪ Design Strategy

▪

### Divide-and-Conquer

▪

### Dynamic Programming

▪

### Greedy Algorithms

▪

### Graph Algorithms

### ▪ Analysis

▪

### Amortized Analysis

**3**

Textbook Chapter 17 – Amortized Analysis

**4**

### ▪ A data structure comes with operations that organize the stored data

▪

### Different operations may have different costs

▪

### The same operation may have different costs

**5**

stack

PUSH POP

MULTIPOP

cost

operations

**6**

stack

PUSH POP

MULTIPOP

cost

operations worst-case

Cost of stack operations PUSH(S, x) = O(1)

POP(S) = O(1)

MULTIPOP(S, k) = O(min(|S|, k))

▪

### n-th operation takes M

^{ULTIPOP}

*(S, n) = O(n) time in the worst case*

▪

*n operations take O(n*

^{2}

### ) time

**7**

**Stack Operations**

*Suppose that we apply a sequence of n operations on a data structure. What is *
the time complexity of the procedure?

Can this be an over-estimate?

*What if only a few operations take O(n) time *
*and the rest of them take O(1) time?*

The worst-case bound is not tight because this

expensive Multipop operation cannot occur so frequently!

▪

*Goal: obtain an accurate worst-case bound in executing a sequence of * *operations on a given data structure*

▪ An upper bound for any *sequence of n operations*

▪

### Comparison: types of running-time analysis

**8**

**Type** **Description**

Worst case **Running time guarantee for any input of size n**

Average case **Expected running time for a random input of size n**
Probabilistic **Expected running time of a randomized algorithm**

Amortized **Worst-case running time for a sequence of n operations**

Aggregate method (聚集法)

• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations

• The average cost per operation is then 𝑇(𝑛)/𝑛

• All operations have the same amortized cost

Accounting method (記帳法)

• Each operation is assigned an amortized cost (may differ from the actual cost)

• Each object of the data structure is associated with a credit

• Need to ensure that every object has sufficient credit at any time

Potential method (位能法)

• Similar to accounting method; each operation is assigned an amortized cost

• The data structure as a whole maintains a credit (i.e., potential)

• Need to ensure that the potential level is nonnegative at any time ^{9}

Textbook Chapter 17.1 – Aggregate analysis

Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method

**10**

▪

### Implementation with an array or a linked list

**11**

**Operation Type** **Cost**

PUSH(S, x): inset an element x into S POP(S): pop the top element from S

MULTIPOP(S, k): pop top k elements from S at once

stack

PUSH POP

MULTIPOP MULTIPOP(S, k)

**while not STACK-EMPTY(S) and k > 0**
POP(S)

k = k - 1

**Stack Operations**

*Suppose that we apply a sequence of n operations on a data structure. What is *
the time complexity of the procedure?

▪

### Approach:

1.

### Determine an upper bound 𝑇(𝑛) on the cost of any sequence of 𝑛 operations

2.

### Calculate the amortized cost per operation as 𝑇(𝑛)/𝑛

3.

**All operations have the same amortized cost**

**12**

cost

operations 𝑇(𝑛) Amortized cost of each op =

op_{n}

op_{1} op_{2} … …

▪

### The number of each operation type

▪

*These n*

_{pop}*+ n*

_{multipop}### operations together take at most

▪

*Total cost for n operations:*

▪

### Amortized cost per operation:

**13**

**Operation Type** **#Operations**

PUSH(S, x): inset an element x into S *n** _{push}*
POP(S): pop the top element from S

*n*

*MULTIPOP(S, k): pop top k elements from S at once*

_{pop}*n*

_{multipop}*n*

Key idea: #pop elements ≤ #push operations/elements

▪

### Once the push operation is taken, we prepare the additional cost for the future usage of multipop

**14**

Key idea: #pop elements ≤ #push operations/elements

▪

### Idea: save credits from the operations that take less cost for future use of operations that take more cost (針對使用花費較低的operations時先存錢 未雨綢繆, 供未來花費較高的operations使用)

▪

### Approach:

1.

*Each operation is assigned a valid amortized cost*

▪ If amortized cost > actual cost, the difference becomes credit (存)

▪ Credit is deposited in an object of the data structure

▪ If amortized cost < actual cost, then withdraw (提) stored credits

**2.**

**Validity check: ensure that every object has sufficient credit for any ** *sequence of n operations*

3.

### Calculate total amortized cost based on individual ones

**15**

▪

**Validity check: ensure that every object has sufficient credit for any times ** *of n operations (不能有赤字)*

▪ c_{i}: the actual cost of the i-th operation

▪ ĉ_{i}: the amortized cost of the i-th operation

*→ For all sequences of n operations, we require*

**16**

▪

**Aggregate Method**

▪ Each type of operations have its actual cost

▪ Compute amortized cost using T(n)

▪

**Accounting Method**

▪ Each type of operations can have a different amortized cost

▪ Assign valid amortized costs first and then compute T(n)

1.

### Assign the amortized cost

2.

### Show that for each object s.t.

▪ PUSH: the pushed element is deposited $1 credit

▪ POP and MULTIPOP: use the credit stored with the popped element

▪ There is always enough credit to pay for each operation

3.

*Each amortized cost is O(1) → total amortized cost is O(n)*

**17**

**Operation Type** **Actual Cost** **Amortized Cost**

PUSH(S, x) 1 2

POP(S) 1 0

MULTIPOP(S, k) min(|S|, k) 0

▪

### Idea: represent the prepaid work as “potential,” which can be released to pay for future operations (the potential is associated with the whole data structure rather than specific objects)

▪

### Approach:

1.

**Select a potential function that takes the current data structure state** as input and outputs a “potential level”

**2.**

**Validity check: ensure that the potential level is nonnegative**

3.

### Calculate the amortized cost of each operation based on the potential function

4.

### Calculate total amortized cost based on individual ones

**18**

▪

**Accounting Method**

▪ Each object within the data structure has its credit

▪

**Potential Method**

▪ The data structure has credits

▪

### Potential function Φ maps any state of the data structure to a real number

▪ D_{0}: the initial state of data structure

▪ D_{i}*: the state of data structure after i-th operation*

▪ c_{i}*: the actual cost of i-th operation*

▪ ĉ_{i}**: the amortized cost of i-th operation, defined as**

**19**

▪

### Total amortized cost

▪

### To obtain an upper bound on the actual cost

▪ *Define a potential function such that*

▪ Usually we set

**20**

1.

### Define Φ 𝐷

_{𝑖}

### to be the number of elements in the stack *after the i-th* operation

2.

### Validity check:

▪ The stack is initially empty →

▪ The number of elements in the stack is always ≥ 0 → 3.

### Compute amortized cost of each operation:

▪ PUSH(S, X):

▪ POP(S):

▪ MULTIPOP(S, k):

4.

*All operations have O(1) amortized cost → total amortized cost is O(n)*

**21**

Practice: justify why it is zero

c_{i}*: the actual cost of i-th operation*
ĉ_{i}*: the amortized cost of i-th operation*

**22**

**23**

▪

### Fibonacci heap (Textbook Ch. 19)

▪ BUILD-MIN-HEAP:

▪ EXTRACT-MIN: (amortized)

▪ DECREASE-KEY: (amortized)

▪

### Total complexity:

MST-PRIM(G, w, r) // w = weights, r = root for u in G.V

u.key = ∞ u.π = NIL r.key = 0 Q = G.V

while Q ≠ empty

u = EXTRACT-MIN(Q) for v in G.adj[u]

if v ∈ Q and w(u, v) < v.key v.π = u

v.key = w(u, v) // DECREASE-KEY

▪

### Fibonacci heap (Textbook Ch. 19)

▪ BUILD-MIN-HEAP:

▪ EXTRACT-MIN: (amortized)

▪ DECREASE-KEY: (amortized)

▪

### Total complexity:

**24**

DIJKSTRA(G, w, s)

INITIALIZATION(G, s) S = empty

Q = G.v // INSERT while Q ≠ empty

u = EXTRACT-MIN(Q) S = S∪{u}

for v in G.adj[u]

RELAX(u, v, w)

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

Textbook Chapter 17.1 – Aggregate analysis

Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method

**25**

▪

*Implementation with a k-bit array*

**26**

INCREMENT(A) i = 0

**while i < A.length and A[i] == 1**
A[i] = 0

i = i + 1
**if i < A.length**

A[i] = 1

**Binary Counter**

*Suppose that a counter is initially zero. We increment the counter n times. How *
many bits are altered throughout the process?

0 1 10 11 100 101 110 111 1000

1001 1010 1011 1100 1101 1110 1111 10000 increment

▪

*Each operation takes O(log n) time in the worst case*

▪

*n operations take O(n log n) time*

**27**

**Counter **

**Value** **A[3]** **A[2]** **A[1]** **A[0]** **Total Cost of First n**

**Operations**

0 0 0 0 0 0

1 0 0 0 **1** 1

2 0 0 **1** **0** 3

3 0 0 1 **1** 4

4 0 **1** **0** **0** 7

5 0 1 0 **1** 8

6 0 1 **1** **0** 10

7 0 1 1 **1** 11

8 **1** **0** **0** **0** 15

flip every increment flip every 2 increments

flip every 4 increments flip every 8 increments

▪

*Total #bits flipping in n increment operations:*

▪

### Total cost of the sequence:

▪

### Amortized cost per operation:

**28**

1.

### Assign the amortized cost

2.

### Validity check:

▪ Each bit 0 to bit 1, we save additional $1 in the bit 1

▪ When bit 1 becomes to bit 0, we spend the saved cost 3.

### Each increment

▪ Change many 1s to 0s → free

▪ *Change exactly a 0 to 1 → O(1)*

▪

*Each amortized cost is O(1) → total amortized cost is O(n)*

**29**

**Operation** **Actual Cost** **Amortized Cost**

bit 0 → bit 1 1 2 (存$1到bit 1)

bit 1 → bit 0 1 0 (用掉存在bit 1裡面的$1)

increment #flipped bits 2 for setting a bit to 1

**30**

**Counter **

**Value** **A[3]** **A[2]** **A[1]** **A[0]** **Total Cost of First n**

**Operations**

0 0 0 0 0 0

1 0 0 0 **1** 1

2 0 0 **1** **0** 3

3 0 0 1 **1** 4

4 0 **1** **0** **0** 7

5 0 1 0 **1** 8

6 0 1 **1** **0** 10

7 0 1 1 **1** 11

8 **1** **0** **0** **0** 15

*Amortized cost per operation is O(1)*

*Total amortized cost of n operations is O(n)*

1.

### Define Φ 𝐷

_{𝑖}

### to be the number of 1s in the counter *after the i-th* operation

2.

### Validity check:

▪ The counter is initially zero →

▪ The number of 1’s cannot be negative →

3.

### Compute amortized cost of each INCREMENT:

▪ Let LSB_{0}*(i) be the number of continuous 1s in the suffix*

▪ For example, LSB_{0}(01011011) = 2, and LSB_{0}(01011111) = 5

4.

*All operations have O(1) amortized cost → total amortized cost is O(n)*

**31**

c_{i}*: the actual cost of i-th operation*
ĉ_{i}*: the amortized cost of i-th operation*

**32**

Aggregate method (聚集法)

• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations

• The average cost per operation is then 𝑇(𝑛)/𝑛

• All operations have the same amortized cost

Accounting method (記帳法)

• Each operation is assigned an amortized cost (may differ from the actual cost)

• Each object of the data structure is associated with a credit

• Need to ensure that every object has sufficient credit at any time

Potential method (位能法)

• Similar to accounting method; each operation is assigned an amortized cost

• The data structure as a whole maintains a credit (i.e., potential)

• Need to ensure that the potential level is nonnegative at any time

Three analyzing methods reach the same answer, and choose your preference

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw

**33**

Important announcement will be sent to @ntu.edu.tw mailbox

& post to the course website