Slides credited from Hsueh-I Lu & Hsu-Chun Hsiao
▪ Amortized analysis
▪ #1: Stack Operations
▪ Aggregate method
▪ Accounting method
▪ Potential method
▪ #2: Binary Counter
▪ Aggregate method
▪ Accounting method
▪ Potential method
2
▪ Design Strategy
▪
Divide-and-Conquer
▪
Dynamic Programming
▪
Greedy Algorithms
▪
Graph Algorithms
▪ Analysis
▪
Amortized Analysis
3
Textbook Chapter 17 – Amortized Analysis
4
▪ A data structure comes with operations that organize the stored data
▪
Different operations may have different costs
▪
The same operation may have different costs
5
stack
PUSH POP
MULTIPOP
cost
operations
6
stack
PUSH POP
MULTIPOP
cost
operations worst-case
Cost of stack operations PUSH(S, x) = O(1)
POP(S) = O(1)
MULTIPOP(S, k) = O(min(|S|, k))
▪
n-th operation takes M
ULTIPOP(S, n) = O(n) time in the worst case
▪
n operations take O(n
2) time
7
Stack Operations
Suppose that we apply a sequence of n operations on a data structure. What is the time complexity of the procedure?
Can this be an over-estimate?
What if only a few operations take O(n) time and the rest of them take O(1) time?
The worst-case bound is not tight because this
expensive Multipop operation cannot occur so frequently!
▪
Goal: obtain an accurate worst-case bound in executing a sequence of operations on a given data structure
▪ An upper bound for any sequence of n operations
▪
Comparison: types of running-time analysis
8
Type Description
Worst case Running time guarantee for any input of size n
Average case Expected running time for a random input of size n Probabilistic Expected running time of a randomized algorithm
Amortized Worst-case running time for a sequence of n operations
Aggregate method (聚集法)
• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations
• The average cost per operation is then 𝑇(𝑛)/𝑛
• All operations have the same amortized cost
Accounting method (記帳法)
• Each operation is assigned an amortized cost (may differ from the actual cost)
• Each object of the data structure is associated with a credit
• Need to ensure that every object has sufficient credit at any time
Potential method (位能法)
• Similar to accounting method; each operation is assigned an amortized cost
• The data structure as a whole maintains a credit (i.e., potential)
• Need to ensure that the potential level is nonnegative at any time 9
Textbook Chapter 17.1 – Aggregate analysis
Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method
10
▪
Implementation with an array or a linked list
11
Operation Type Cost
PUSH(S, x): inset an element x into S POP(S): pop the top element from S
MULTIPOP(S, k): pop top k elements from S at once
stack
PUSH POP
MULTIPOP MULTIPOP(S, k)
while not STACK-EMPTY(S) and k > 0 POP(S)
k = k - 1
Stack Operations
Suppose that we apply a sequence of n operations on a data structure. What is the time complexity of the procedure?
▪
Approach:
1.
Determine an upper bound 𝑇(𝑛) on the cost of any sequence of 𝑛 operations
2.
Calculate the amortized cost per operation as 𝑇(𝑛)/𝑛
3.
All operations have the same amortized cost
12
cost
operations 𝑇(𝑛) Amortized cost of each op =
opn
op1 op2 … …
▪
The number of each operation type
▪
These n
pop+ n
multipopoperations together take at most
▪
Total cost for n operations:
▪
Amortized cost per operation:
13
Operation Type #Operations
PUSH(S, x): inset an element x into S npush POP(S): pop the top element from S npop MULTIPOP(S, k): pop top k elements from S at once nmultipop
n
Key idea: #pop elements ≤ #push operations/elements
▪
Once the push operation is taken, we prepare the additional cost for the future usage of multipop
14
Key idea: #pop elements ≤ #push operations/elements
▪
Idea: save credits from the operations that take less cost for future use of operations that take more cost (針對使用花費較低的operations時先存錢 未雨綢繆, 供未來花費較高的operations使用)
▪
Approach:
1.
Each operation is assigned a valid amortized cost
▪ If amortized cost > actual cost, the difference becomes credit (存)
▪ Credit is deposited in an object of the data structure
▪ If amortized cost < actual cost, then withdraw (提) stored credits
2.
Validity check: ensure that every object has sufficient credit for any sequence of n operations
3.
Calculate total amortized cost based on individual ones
15
▪
Validity check: ensure that every object has sufficient credit for any times of n operations (不能有赤字)
▪ ci: the actual cost of the i-th operation
▪ ĉi: the amortized cost of the i-th operation
→ For all sequences of n operations, we require
16
▪
Aggregate Method
▪ Each type of operations have its actual cost
▪ Compute amortized cost using T(n)
▪
Accounting Method
▪ Each type of operations can have a different amortized cost
▪ Assign valid amortized costs first and then compute T(n)
1.
Assign the amortized cost
2.
Show that for each object s.t.
▪ PUSH: the pushed element is deposited $1 credit
▪ POP and MULTIPOP: use the credit stored with the popped element
▪ There is always enough credit to pay for each operation
3.
Each amortized cost is O(1) → total amortized cost is O(n)
17
Operation Type Actual Cost Amortized Cost
PUSH(S, x) 1 2
POP(S) 1 0
MULTIPOP(S, k) min(|S|, k) 0
▪
Idea: represent the prepaid work as “potential,” which can be released to pay for future operations (the potential is associated with the whole data structure rather than specific objects)
▪
Approach:
1.
Select a potential function that takes the current data structure state as input and outputs a “potential level”
2.
Validity check: ensure that the potential level is nonnegative
3.
Calculate the amortized cost of each operation based on the potential function
4.
Calculate total amortized cost based on individual ones
18
▪
Accounting Method
▪ Each object within the data structure has its credit
▪
Potential Method
▪ The data structure has credits
▪
Potential function Φ maps any state of the data structure to a real number
▪ D0: the initial state of data structure
▪ Di: the state of data structure after i-th operation
▪ ci: the actual cost of i-th operation
▪ ĉi: the amortized cost of i-th operation, defined as
19
▪
Total amortized cost
▪
To obtain an upper bound on the actual cost
▪ Define a potential function such that
▪ Usually we set
20
1.
Define Φ 𝐷
𝑖to be the number of elements in the stack after the i-th operation
2.
Validity check:
▪ The stack is initially empty →
▪ The number of elements in the stack is always ≥ 0 → 3.
Compute amortized cost of each operation:
▪ PUSH(S, X):
▪ POP(S):
▪ MULTIPOP(S, k):
4.
All operations have O(1) amortized cost → total amortized cost is O(n)
21
Practice: justify why it is zero
ci: the actual cost of i-th operation ĉi: the amortized cost of i-th operation
22
23
▪
Fibonacci heap (Textbook Ch. 19)
▪ BUILD-MIN-HEAP:
▪ EXTRACT-MIN: (amortized)
▪ DECREASE-KEY: (amortized)
▪
Total complexity:
MST-PRIM(G, w, r) // w = weights, r = root for u in G.V
u.key = ∞ u.π = NIL r.key = 0 Q = G.V
while Q ≠ empty
u = EXTRACT-MIN(Q) for v in G.adj[u]
if v ∈ Q and w(u, v) < v.key v.π = u
v.key = w(u, v) // DECREASE-KEY
▪
Fibonacci heap (Textbook Ch. 19)
▪ BUILD-MIN-HEAP:
▪ EXTRACT-MIN: (amortized)
▪ DECREASE-KEY: (amortized)
▪
Total complexity:
24
DIJKSTRA(G, w, s)
INITIALIZATION(G, s) S = empty
Q = G.v // INSERT while Q ≠ empty
u = EXTRACT-MIN(Q) S = S∪{u}
for v in G.adj[u]
RELAX(u, v, w)
INITIALIZATION(G, s) for v in G.V
v.d = ∞ v.π = NIL s.d = 0
RELAX(u, v, w)
if v.d > u.d + w(u, v) // DECREASE-KEY
v.d = u.d + w(u, v) v.π = u
Textbook Chapter 17.1 – Aggregate analysis
Textbook Chapter 17.2 – The accounting method Textbook Chapter 17.3 – The potential method
25
▪
Implementation with a k-bit array
26
INCREMENT(A) i = 0
while i < A.length and A[i] == 1 A[i] = 0
i = i + 1 if i < A.length
A[i] = 1
Binary Counter
Suppose that a counter is initially zero. We increment the counter n times. How many bits are altered throughout the process?
0 1 10 11 100 101 110 111 1000
1001 1010 1011 1100 1101 1110 1111 10000 increment
▪
Each operation takes O(log n) time in the worst case
▪
n operations take O(n log n) time
27
Counter
Value A[3] A[2] A[1] A[0] Total Cost of First n
Operations
0 0 0 0 0 0
1 0 0 0 1 1
2 0 0 1 0 3
3 0 0 1 1 4
4 0 1 0 0 7
5 0 1 0 1 8
6 0 1 1 0 10
7 0 1 1 1 11
8 1 0 0 0 15
flip every increment flip every 2 increments
flip every 4 increments flip every 8 increments
▪
Total #bits flipping in n increment operations:
▪
Total cost of the sequence:
▪
Amortized cost per operation:
28
1.
Assign the amortized cost
2.
Validity check:
▪ Each bit 0 to bit 1, we save additional $1 in the bit 1
▪ When bit 1 becomes to bit 0, we spend the saved cost 3.
Each increment
▪ Change many 1s to 0s → free
▪ Change exactly a 0 to 1 → O(1)
▪
Each amortized cost is O(1) → total amortized cost is O(n)
29
Operation Actual Cost Amortized Cost
bit 0 → bit 1 1 2 (存$1到bit 1)
bit 1 → bit 0 1 0 (用掉存在bit 1裡面的$1)
increment #flipped bits 2 for setting a bit to 1
30
Counter
Value A[3] A[2] A[1] A[0] Total Cost of First n
Operations
0 0 0 0 0 0
1 0 0 0 1 1
2 0 0 1 0 3
3 0 0 1 1 4
4 0 1 0 0 7
5 0 1 0 1 8
6 0 1 1 0 10
7 0 1 1 1 11
8 1 0 0 0 15
Amortized cost per operation is O(1)
Total amortized cost of n operations is O(n)
1.
Define Φ 𝐷
𝑖to be the number of 1s in the counter after the i-th operation
2.
Validity check:
▪ The counter is initially zero →
▪ The number of 1’s cannot be negative →
3.
Compute amortized cost of each INCREMENT:
▪ Let LSB0(i) be the number of continuous 1s in the suffix
▪ For example, LSB0(01011011) = 2, and LSB0(01011111) = 5
4.
All operations have O(1) amortized cost → total amortized cost is O(n)
31
ci: the actual cost of i-th operation ĉi: the amortized cost of i-th operation
32
Aggregate method (聚集法)
• Determine an upper bound 𝑇(𝑛) on the cost over any sequence of 𝑛 operations
• The average cost per operation is then 𝑇(𝑛)/𝑛
• All operations have the same amortized cost
Accounting method (記帳法)
• Each operation is assigned an amortized cost (may differ from the actual cost)
• Each object of the data structure is associated with a credit
• Need to ensure that every object has sufficient credit at any time
Potential method (位能法)
• Similar to accounting method; each operation is assigned an amortized cost
• The data structure as a whole maintains a credit (i.e., potential)
• Need to ensure that the potential level is nonnegative at any time
Three analyzing methods reach the same answer, and choose your preference
Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw
33
Important announcement will be sent to @ntu.edu.tw mailbox
& post to the course website
