**Algorithm Design and Analysis** **Amortized Analysis**

Yun-Nung (Vivian) Chen

**http://ada.miulab.tw**

**Outline**

• Amortized analysis

• #1: Stack Operations

• Aggregate method

• Accounting method

• Potential method

• #2: Binary Counter

• Aggregate method

• Accounting method

• Potential method

**Algorithm Design & Analysis**

### • Design Strategy

• Divide-and-Conquer

• Dynamic Programming

• Greedy Algorithms

• Graph Algorithms

### • Analysis

• Amortized Analysis

**Amortized Analysis**

Textbook Chapter 17 – Amortized Analysis

**Data-Structure Operations**

### • A data structure comes with operations that organize the stored data

• Different operations may have different costs

• The same operation may have different costs

stack

PUSH POP

MULTIPOP

cost

operations

**Worst Case Time Complexity**

stack

PUSH POP

MULTIPOP

cost

operations worst-case

Cost of stack operations
P^{USH}(S, x) = O(1)

P^{OP}(S) = O(1)

M^{ULTIPOP}(S, k) = O(min(|S|, k))

**Worst Case Time Complexity**

• n-th operation takes M^{ULTIPOP}*(S, n) = O(n) time in the worst case*

*• n operations take O(n*^{2}) time

**Stack Operations**

*Suppose that we apply a sequence of n operations on a data structure. What is the *
time complexity of the procedure?

Can this be an over-estimate?

*What if only a few operations take O(n) time and *
*the rest of them take O(1) time?*

The worst-case bound is not tight because this

expensive Multipop operation cannot occur so frequently!

**Amortized Analysis**

*• Goal: obtain an accurate worst-case bound in executing a sequence of *
*operations on a given data structure*

• An upper bound for any *sequence of n operations*

• Comparison: types of running-time analysis

**Type** **Description**

Worst case **Running time guarantee for any input of size n**

Average case **Expected running time for a random input of size n**
Probabilistic **Expected running time of a randomized algorithm**

Amortized **Worst-case running time for a sequence of n operations**

**3 Methods for Amortized Analysis**

## Aggregate method (聚集法)

• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations

• The average cost per operation is then 𝑇(𝑛)/𝑛

• All operations have the same amortized cost

Accounting method (記帳法)

• Each operation is assigned an amortized cost (may differ from the actual cost)

• Each object of the data structure is associated with a credit

• Need to ensure that every object has sufficient credit at any time

Potential method (位能法)

• Similar to accounting method; each operation is assigned an amortized cost

• The data structure as a whole maintains a credit (i.e., potential)

• Need to ensure that the potential level is nonnegative at any time

**Stack Operations**

Textbook Chapter 17.1 – Aggregate analysis

Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method

**Stack Operations**

• Implementation with an array or a linked list

**Operation Type** **Cost**

PUSH(S, x): inset an element x into S
P^{OP}(S): pop the top element from S

M^{ULTIPOP}(S, k): pop top k elements from S at once

stack

PUSH POP

MULTIPOP MULTIPOP(S, k)

**while not STACK-EMPTY(S) and k > 0**
POP(S)

k = k - 1

**Stack Operations**

*Suppose that we apply a sequence of n operations on a data structure. What is the *
time complexity of the procedure?

**Aggregate Method (聚集法)**

• Approach:

1. Determine an upper bound 𝑇(𝑛) on the cost of any sequence of 𝑛 operations 2. Calculate the amortized cost per operation as 𝑇(𝑛)/𝑛

**3. All operations have the same amortized cost**

cost

operations 𝑇(𝑛) Amortized cost of each op =

… …

**Aggregate Method for Stack**

• The number of each operation type

*• These n*_{pop}*+ n** _{multipop}* operations together take at most

*• Total cost for n operations:*

• Amortized cost per operation:

**Operation Type** **#Operations**

P^{USH}(S, x): inset an element x into S *n** _{push}*
P

^{OP}(S): pop the top element from S

*n*

*MULTIPOP(S, k): pop top k elements from S at once*

_{pop}*n*

_{multipop}*n*

Key idea: #pop elements ≤ #push operations/elements

**Another Thinking**

• Once the push operation is taken, we prepare the additional cost for the future usage of multipop

Key idea: #pop elements ≤ #push operations/elements

**Accounting Method (記帳法)**

• Idea: save credits from the operations that take less cost for future use of

operations that take more cost (針對使用花費較低的operations時先存錢未 雨綢繆, 供未來花費較高的operations使用)

• Approach:

*1. Each operation is assigned a valid amortized cost*

• If amortized cost > actual cost, the difference becomes credit (存)

• Credit is deposited in an object of the data structure

• If amortized cost < actual cost, then withdraw (提) stored credits

* 2. Validity check: ensure that every object has sufficient credit for any sequence of n*
operations

3. Calculate total amortized cost based on individual ones

**Accounting Method (記帳法)**

**• Validity check: ensure that every object has sufficient credit for any times of **
*n operations (不能有赤字)*

• c_{i}: the actual cost of the i-th operation

• ĉ_{i}: the amortized cost of the i-th operation

*→ For all sequences of n operations, we require*

▪ **Aggregate Method**

▪ Each type of operations have its actual cost

▪ Compute amortized cost using T(n)

▪ **Accounting Method**

▪ Each type of operations can have a different amortized cost

▪ Assign valid amortized costs first and then compute T(n)

**Accounting Method for Stack**

1. Assign the amortized cost

2. Show that for each object s.t.

• P^{USH}: the pushed element is deposited $1 credit

• P^{OP} and M^{ULTIPOP}: use the credit stored with the popped element

• There is always enough credit to pay for each operation

*3. Each amortized cost is O(1) → total amortized cost is O(n)*

**Operation Type** **Actual Cost** **Amortized Cost**

PUSH(S, x) 1 2

POP(S) 1 0

MULTIPOP(S, k) min(|S|, k) 0

**Potential Method (位能法)**

• Idea: represent the prepaid work as “potential,” which can be released to pay for future operations (the potential is associated with the whole data

structure rather than specific objects)

• Approach:

**1. Select a potential function that takes the current data structure state as input and **
outputs a “potential level”

**2. Validity check: ensure that the potential level is nonnegative**

3. Calculate the amortized cost of each operation based on the potential function 4. Calculate total amortized cost based on individual ones

▪ **Accounting Method**

▪ Each object within the data structure has its credit

▪ **Potential Method**

▪ The data structure has credits

**Potential Method (位能法)**

• Potential function Φ maps any state of the data structure to a real number

• D_{0}: the initial state of data structure

• D_{i}*: the state of data structure after i-th operation*

• c_{i}*: the actual cost of i-th operation*

• ĉ_{i}**: the amortized cost of i-th operation, defined as**

**Potential Method (位能法)**

• Total amortized cost

• To obtain an upper bound on the actual cost

*• Define a potential function such that*

• Usually we set

**Potential Method for Stack**

1. Define Φ 𝐷_{𝑖} to be the number of elements in the stack *after the i-th*
operation

2. Validity check:

• The stack is initially empty →

• The number of elements in the stack is always ≥ 0 →

3. Compute amortized cost of each operation:

• P^{USH}(S, X):

• P^{OP}(S):

• M^{ULTIPOP}(S, k):

*4. All operations have O(1) amortized cost → total amortized cost is O(n)*

Practice: justify why it is zero

c_{i}*: the actual cost of i-th operation*
ĉ_{i}*: the amortized cost of i-th operation*

**Fibonacci Heap**

**Prim’s Time Complexity**

• Fibonacci heap (Textbook Ch. 19)

• BUILD-MIN-HEAP:

• EXTRACT-MIN: (amortized)

• DECREASE-KEY: (amortized)

• Total complexity:

MST-PRIM(G, w, r) // w = weights, r = root for u in G.V

u.key = ∞ u.π = NIL r.key = 0 Q = G.V

while Q ≠ empty

u = EXTRACT-MIN(Q) for v in G.adj[u]

if v ∈ Q and w(u, v) < v.key v.π = u

v.key = w(u, v) // DECREASE-KEY

**Dijkstra’s Time Complexity**

• Fabonacci heap (Textbook Ch. 19)

• BUILD-MIN-HEAP:

• EXTRACT-MIN: (amortized)

• DECREASE-KEY: (amortized)

• Total complexity:

DIJKSTRA(G, w, s)

INITIALIZATION(G, s) S = empty

Q = G.v // INSERT while Q ≠ empty

u = EXTRACT-MIN(Q) S = S∪{u}

for v in G.adj[u]

RELAX(u, v, w)

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

**Binary Counter**

Textbook Chapter 17.1 – Aggregate analysis

Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method

**Binary Counter**

*• Implementation with a k-bit array*

*• Each operation takes O(log n) time in the worst case*

*• n operations take O(n log n) time*

INCREMENT(A) i = 0

**while i < A.length and A[i] == 1**
A[i] = 0

i = i + 1

**if i < A.length**
A[i] = 1

**Binary Counter**

*Suppose that a counter is initially zero. We increment the counter n times. How *
many bits are altered throughout the process?

0 1 10 11 100 101 110 111 1000

1001 1010 1011 1100 1101 1110 1111 10000 increment

**Aggregate Method for Binary Counter**

**Counter **

**Value** **A[3]** **A[2]** **A[1]** **A[0]** **Total Cost of First n****Operations**

0 0 0 0 0 0

1 0 0 0 **1** 1

2 0 0 **1** **0** 3

3 0 0 1 **1** 4

4 0 **1** **0** **0** 7

5 0 1 0 **1** 8

6 0 1 **1** **0** 10

7 0 1 1 **1** 11

8 **1** **0** **0** **0** 15

flip every increment flip every 2 increments

flip every 4 increments flip every 8 increments

**Aggregate Method for Binary Counter**

*• Total #bits flipping in n increment operations:*

• Total cost of the sequence:

• Amortized cost per operation:

**Accounting Method for Binary Counter**

1. Assign the amortized cost

2. Validity check:

• Each bit 0 to bit 1, we save additional $1 in the bit 1

• When bit 1 becomes to bit 0, we spend the saved cost

3. Each increment

• Change many 1s to 0s → free

*• Change exactly a 0 to 1 → O(1)*

*• Each amortized cost is O(1) → total amortized cost is O(n)*

**Operation** **Actual Cost** **Amortized Cost**

bit 0 → bit 1 1 2 (存$1到bit 1)

bit 1 → bit 0 1 0 (用掉存在bit 1裡面的$1)

increment #flipped bits 2 for setting a bit to 1

**Accounting Method for Binary Counter**

**Counter **

**Value** **A[3]** **A[2]** **A[1]** **A[0]** **Total Cost of First n**

**Operations**

0 0 0 0 0 0

1 0 0 0 **1** 1

2 0 0 **1** **0** 3

3 0 0 1 **1** 4

4 0 **1** **0** **0** 7

5 0 1 0 **1** 8

6 0 1 **1** **0** 10

7 0 1 1 **1** 11

8 **1** **0** **0** **0** 15

**Potential Method for Binary Counter**

1. Define Φ 𝐷_{𝑖} to be the number of 1s in the counter *after the i-th*
operation

2. Validity check:

• The counter is initially zero →

• The number of 1’s cannot be negative →

3. Compute amortized cost of each INCREMENT:

• Let LSB_{0}*(i) be the number of continuous 1s in the suffix*

• For example, LSB_{0}(01011011) = 2, and LSB_{0}(01011111) = 5

*4. All operations have O(1) amortized cost → total amortized cost is O(n)*

c_{i}*: the actual cost of i-th operation*
ĉ_{i}*: the amortized cost of i-th operation*

**Concluding Remarks**

Aggregate method (聚集法)

• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations

• The average cost per operation is then 𝑇(𝑛)/𝑛

• All operations have the same amortized cost

Accounting method (記帳法)

• Each operation is assigned an amortized cost (may differ from the actual cost)

• Each object of the data structure is associated with a credit

• Need to ensure that every object has sufficient credit at any time

Potential method (位能法)

• Similar to accounting method; each operation is assigned an amortized cost

• The data structure as a whole maintains a credit (i.e., potential)

• Need to ensure that the potential level is nonnegative at any time

## Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw