**Chapter 17 **

**Amortized Analysis**

**Lee, Hsiu-Hui**

**Ack: This presentation is based on the **
**lecture slides from Hsu, Lih-Hsing, as **
**well as various materials from the web.**

**Amortized Analysis**

### • **The time required to perform a sequence of ** **data structure operations is averaged over all ** **the operations performed.**

### • **An amortized analysis guarantees **

**the average performance of each operation**

**in the worst case.**

**Three common techniques **

### • **aggregate analysis**

### • **accounting method**

### • **potential method**

**17.1 Aggregate Analysis**

### • **For all n, a sequence of n operations takes **

**For all n, a sequence of n operations takes**

**worst time T(n) in total. The amortize cost of ** **each operation is .**

**worst time T(n) in total. The amortize cost of**

**n****n*** T*( )

**EX1-Stack operation**

**PUSH(S, x) / POP(S) / ** **MULTIPOP(S, k)**

**PUSH(S, x) / POP(S) /**

**MULTIPOP(S, k)**

**MULTIPOP(S, k)**

**1** **while not STACK-EMPTY(S) and k ≠ 0**

**2** **do POP(S)**

**3** **k****← k – 1 **

**Action of M** **ULTIPOP ** **on a ** **stack S**

**MULTIPOP(S,4)** **MULTIPOP(S,7)**

top → 23 17 6 39 10 47

⎯⎯

top → 10 47

⎯⎯ ⎯⎯

**• PUSH(S, x) / POP(S)**

**• PUSH(S, x) / POP(S)**

**Each runs in O(1) time.**

**Actual running time for a sequence of n PUSH, ****POP operations is **Θ^{(n).}

**• MULTIPOP(S, k)**

**• MULTIPOP(S, k)**

**pops the top k objects of stack **

**Total cost : min(s, k) s : stack size**

### • **Analyze a sequence of n PUSH, POP, and **

**Analyze a sequence of n PUSH, POP, and**

**MULTIPOP operation on an initially empty stack.**

**問每一個 operation 在 worst-case 之 amortized cost?**

**PUSH** **O(1)**

**POP ** **O(1)**

**MULTIPOP** **O(n) (the stack size is at most n)**

**Total cost of n operations: O(n**

**Total cost of n operations: O(n**

^{2}**)**

**• We can get a better bound. **

**(see the next)**

**Total cost of any seq of n operations: O(n) **

**Total cost of any seq of n operations: O(n)**

### • **Each object can be popped at most once for each ** **time it is pushed.**

**Number of times the POP can be called on a nonempty **
**stack, including calls within MULTIPOP, is at most **
**the number of PUSH operatons, which is at most n.**

**Total cost of any seq of n operations: O(n) **

**Total cost of any seq of n operations: O(n)**

**better bound !**

**The amortized cost of an operation is**

^{(}

^{)}

_{O}_{(}

_{1}

_{)}

^{.}**n****n*** O* =

**EX2-Incrementing a binary counter**

**A[0] 每 1 次改變一次**
**A[1] 每 2 次改變一次**
**A[i] 每 2**** ^{i}** 次改變一次

**I** **NCREMENT** **(A)** **1 i** **← 0**

**(A)**

**1 i**

**2 while i < length[A] and A[i] = 1** **3** **do A[i] ← 0**

**2 while i < length[A] and A[i] = 1**

**do A[i] ← 0**

**4** **i** **← i + 1**

**i**

**← i + 1**

**5 if i < length[A]**

**5 if i < length[A]**

**6** **then A[i] ← 1**

**then A[i] ← 1**

**length[A]=k*** Cost :* Θ

**(k)****Analysis: **

### • **A seq of n increment operations : O(nk) ** **(k is the word length)**

**A seq of n increment operations : O(nk)**

**(k is the word length)**

### • **Amortize Analysis:**

**The total number of flips in the sequence is**

⎣ ⎦

**)**
**)** **(**

**is** **(**
**operation**
**per**

**cost **
**amortized**

**the**

** **

1 2 2

1

2 _{0}

log

0

**n****O****n****O****n**

**n****n**

**i****i****n**

**i**

**i**

=

⇒

=

⎥⎦ <

⎢⎣ ⎥

⎢

### ∑

### ∑

^{∞}

=

=

**17.2 Accounting method**

### • **We assign differing charges to different ** **operations, with some operations charged **

**more or less than the actual cost. The amount ** **we charge an operation is called its amortized ** **cost. **

**Some are charged more than actual cost.**

**Some are charged less.**

### • **(在aggregate method所有operation有相同的**

**amortized cost)**

### • **If the amortized cost > actual cost, the ** **difference is treated as credit**

### • **Credit can be used later on to help pay for **

**operations whose amortized cost is less than **

**their actual cost.**

### • **If we want analysis with amortized costs to ** **show that in the worst cast the average cost ** **per operation is small, the total amortized cost ** **of a sequence of operations must be an upper ** **bound** **on the total actual cost of the sequence. **

### • **Moreover, as in aggregate analysis, this **

**relationship must hold for all sequences of **

**operations. **

• **If we denote the actual cost** **of the ith operation by c**_{i}**and the amortized cost** **of the ith operation by , we ****require **

**(i.e. total amortized cost ≥ total actual cost)**

**for all sequence of n operations.**

• **The total credit stored in the data structure is the **
**difference between the total actual cost, or**

• **The total credit must be nonnegative at all times.**

*cˆ*

*i*

### ∑

### ∑

= =≥ ^{n}

**n****i****n**

**i**

**i****c**

**c**

1 1

ˆ

### ∑

### ∑

= =− ^{n}

**i**

**i****n**

**i**

**i****c**

**c**

1 1

ˆ

**EX1-Stack operation**

**n 個運作之後, total amortized cost = O(n)**

**0**
**min{k,s}**

**M****ULTIPOP**

**0**
**1**

**POP**

**2**
**1**

**PUSH**

**actual **

**cost** **amortized **
**cost**

**EX2-Incrementing a binary ** **counter**

• **Each time, there is exactly one 0 that is changed **
**into 1.**

• **The number of 1’s in the counter is never negative!**

• **Amortized cost is at most 2 = O(1).**

• **執行 n 次 Increment 其 total amortized cost = O(n)**

**0**
**1→0** **1**

**2**
**0→1** **1**

**actual cost** **amortized cost**

**17.3 Potential method**

### • **Like the accounting method, but think of ** **the credit as potential stored with the **

**entire data structure.**

• **Accounting method stores credit with **
**specific objects.**

• **Potential method stores potential in the data **
**structure as a whole.**

• **Can release potential to pay for future **
**operations.**

• **Most flexible of the amortized analysis **
**methods.**

### z

*0*

**D**### : initial data structure

*z D*

_{i}*: the data structure of the result after applying the i-th * *operation to the data structure D*

_{i}

_{−1}### .

*z C*

_{i}*: actual cost of the i-th operation. *

### z *$c* _{i} *: amotized cost of the i-th operation. *

_{i}

### z A potential function Φ maps each data structure *D* _{i} to a real number

_{i}

*Φ(* *D* _{i} *)* , which is the potential associated with data structure *D* _{i} .

_{i}

_{i}

### z The amortized cost _{$c} _{i} * of the i-th *

_{$c}

_{i}

### operation with respect to potential Φ

### is defined by

**c**^{ˆ}

*=*

_{i}

**c***+Φ*

_{i}^{(}

**D**

_{i}^{)}−Φ

^{(}

**D***−1*

_{i}^{)}

### .

### • **If for all i,**

**If for all i,**

**then the amortized cost is always an upper bound on **
**actual cost .**

### • **If then the potential increases.**

### ∑

### ∑

### ∑

=

= −

=

Φ

− Φ

+

=

Φ

− Φ

+

=

*n*

*i*

*n*
*i*

*n*

*i*

*i*
*i*

*i*
*n*

*i*

*i*

*D*
*D*

*c*

*D*
*D*

*c*
*c*

1

0 1

1 1

) (

) (

) ) (

) ( ˆ (

Φ*(* *D*_{i}*)* ≥ Φ*(* *D*_{0}*)*

*$c*

_{i}*c*

*i*
*n*

*i*
*i*

*n*

= =

### ∑ ≥ ∑

*1* *1*

Φ*(* *D*_{i}*)* ≥ Φ*(* *D*_{i}_{−1}*)*

**EX1-Stack operation**

### • **the number of objects in the ** **stack of the ith operation.**

**stack of the ith operation.**

### •

### •

*Φ(* *D* _{i} *)* =

_{i}

*Φ(* *D* _{0} *)* = *0*

_{0}

*Φ(* *D* _{i} *)* *≥ 0*

_{i}

### PUSH

### Φ *(* *D* _{i} *)* − Φ *(* *D* _{i} _{−1} *)* = *(* *s* + − = *1* *)* *s* *1*

_{i}

_{i}

_{−1}

### $ *(* *)* *(* *)*

*c* _{i} = *c* _{i} + Φ *D* _{i} − Φ *D* _{i} _{−} _{1} = + = *1 1* *2*

_{i}

_{i}

_{i}

_{i}

_{1}

### MULTIPOP

*k* *'* = *min{ , }* *k s*

### Φ *(* *D* _{i} *)* − Φ *(* *D* _{i} _{−1} *)* = − *k* *'*

_{i}

_{i}

_{−1}

### $ *(* *)* *(* *)* *'* *'*

*c* _{i} = *c* _{i} + Φ *D* _{i} − Φ *D* _{i} _{−} _{1} = − = *k* *k* *0*

_{i}

_{i}

_{i}

_{i}

_{1}

### POP

**• The amortized cost of each of these three ** **operations is O(1).**

**operations is O(1).**

**• the total amortized cost of a sequence of **

*$c* _{i} *= 0*

_{i}

**EX2-Incrementing a binary counter**

•

**the number of 1’s in the counter after the ith operations**

• **The ith INCREMENT operation resets bits.**

### •

• **The Counter starts at zero**
**Amortized cost = O(1)**

**Total amortized cost of n operations is O(n)**

**i**

**i**

**b**

**b**

**D** = Φ ( )

**D**

*t*

_{i}### 1 )

### ( = ≤

_{1}

### − + Φ *D*

_{i}*b*

_{i}*b*

_{i}_{−}

*t*

_{i}*i*
*i*

*i*
*i*

*i*

*i*

*D* *b* *t* *b* *t*

*D* − Φ ≤ − + − = −

### Φ ( ) (

_{−}

_{1}

### ) (

_{−}

_{1}

### 1 )

_{−}

_{1}

### 1

### 2 )

### 1 ( )

### 1 (

### ) (

### ) (

### ˆ

_{i}### = *c*

_{i}### + Φ *D*

_{i}### − Φ *D*

_{i}_{−}

_{1}

### ≤ *t*

_{i}### + + − *t*

_{i}### =

*c*

**Even if the counter does ** **not start at zero:**

### 0 0

### 1

### 0 1

### 1

### 2 2

### ) (

### ) ˆ (

*b* *b*

*n* *b*

*b*

*D* *D*

*c* *c*

*n* *n*

*n* *i*

*n* *n*

*i*

*i* *n*

*i*

*i*

### +

### −

### = +

### −

### ≤

### Φ +

### Φ

### −

### =

### ∑

### ∑

### ∑

### =

### =

### =

) (

0

**n****O****k**

**k****b**

**,**
**as**

**long**
**as**

**,**
**since**

**that **
**particular**

**in**
**Note**

=

≤

**17.4 Dynamic tables**

### • **A nice use of amortized analysis.**

### • Scenario

• **Have a table—maybe a hash table.**

• **Don’t know in advance how many objects will be **
**stored in it.**

• **When it fills, must reallocate** **with a larger size, **
**copying all objects into the new larger table.**

• **When it gets sufficiently small, might want to ****reallocate with a smaller size.**

### • **Details of table organization not important.**

### • Goals

• **1. O(1) amortized time per operation.**

• **2. Unused space always ≤ constant fraction **
**of allocated space.**

### • *Load factor * α **= num** / **size, **

**= num**

**size,**

• **num = # items stored**

• **size = allocated size.**

### • **If size = 0, then num = 0. Call ** α **= 1.**

**If size = 0, then num = 0. Call**

### • **Never allow ** α **> 1.**

### • **Keep** α **> a constant fraction ⇒ goal (2).**

**Table expansion**

**• Consider only TABLE-INSERT**

**• TABLE-DELETE (discuss later)**

**• Guarantees that load-factor **

### • **(load factor)**

### • **Each time we actually insert an item into ** **the table, it’s an elementary insertion.**

**the table, it’s an elementary insertion.**

### 2 ) 1

### ( ≥ **α T**

**α T**

### α *( )* *[ ]* *T* *num T* *[ ]*

*size T*

### =

**TABLE_INSERT****(T, x)****1** **if size[T] = 0**

**2** **then allocate table[T] with 1 slot****3** **size[T] ← 1**

**4** **if num[T] = size[T]**

**5** **then allocate new-table with 2 ⋅ size[T] slots****6** **insert all items in table[T] in new-table****7** **free table[T]**

**8** **table[T] ← new-table****9** **size[T]****← 2 ⋅ size[T]**

**10 insert x into table[T]**

**11 num[T] ← num[T] + 1**

**Analysis**

### • **Running time: Charge 1 per elementary **

**Running time: Charge 1 per elementary**

**insertion. Count only elementary insertions, ** **since all other costs together are constant per ** **call.**

### • **c**

**c**

_{i}**= actual cost of ith operation**

**= actual cost of ith operation**

• **If not full, c**_{i}**= 1.**

• **If full, have i - 1 items in the table at the start of ****the ith operation. Have to copy all i - 1 existing ****items, then insert ith item⇒c**_{i}**= i .**

### • **n operations ⇒c**

**n operations ⇒c**

_{i}**= O(n)⇒ O(n**

**= O(n)⇒ O(n**

^{2}**) time for n **

**) time for n**

**operations. (?)**

**Aggregate method:**

**• amortized cost = 3**

### ⎩ ⎨

### = ⎧

### otherwise 1

### 2 of power exact

### an is

### 1 *if i-* *c*

_{i}*i*

⎣ ⎦

### ∑ ∑

=^{n}

### = +

=### < + =

*i*

*n*
*j*

*j*

*i*

*n* *n* *n* *n*

*c*

1

lg

1

### 2 2 3

**Accounting method:**

**Each item pays for 3 elementary insertions;**

**1. inserting itself in the current table,**

**2. moving itself when the table is expanded, and**

**3. moving another item that has already been moved **
**once when the table is expanded.**

**Charge $3 per insertion of x.**

**$1 pays for x’s insertion.**

**$1 pays for x to be moved in the future.**

**$1 pays for some other item to be moved.**

**Potential method:**

### • **(not expansion)**

### ] [

### ] [

### 2 )

### ( *T* = ⋅ *num* *T* − *size* *T* Φ

*size* _{i} = *size* _{i} _{−1}

_{i}

_{i}

_{−1}

### 3

### ) )

### 1 (

### 2 ( )

### 2 ( 1

### ) 2

### ( )

### 2 ( 1

### ˆ

1 1

1

### =

### −

### −

### −

### −

### ⋅ +

### =

### −

### ⋅

### −

### −

### ⋅ +

### =

### Φ

### − Φ +

### =

−

−

−

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*size* *num*

*size* *num*

*size* *num*

*size* *num*

*c*

*c*

**Example:**

**Example:**

*size* _{i} = *size* _{i} _{−1} = *16* *,* *num* _{i} = *13*

_{i}

_{i}

_{−1}

_{i}

### 3

### ) 16 12

### 2 ( )

### 16 13

### 2 ( 1

### ) 16 12

### 2 ( )

### 16 13

### 2 ( 1

### ˆ

_{1}

### =

### −

### ⋅

### −

### −

### ⋅ +

### =

### −

### ⋅

### −

### −

### ⋅ +

### =

### Φ

### − Φ

### +

### =

_{i}

_{i}

_{i}_{−}

*i*

*c*

*c*

### • ^{size}

^{size}

^{i}^{/ 2} ^{=} ^{size}

^{/ 2}

^{size}

^{i}^{−}

^{1}^{=} ^{num}

^{num}

^{i}^{−} ^{1} **(expansion)**

^{1}

### 3

### ) 1 (

### 2

### )) 1 (

### ) 1 (

### 2 (

### )) 2 2

### ( 2

### (

### ) 2

### ( ) 2

### ( ˆ

1 1

1

### =

### −

### − +

### =

### −

### −

### −

### −

### −

### ⋅

### −

### ⋅ +

### =

### −

### ⋅

### −

### −

### ⋅ +

### =

### Φ

### − Φ +

### =

−

−

−

*i*
*i*

*i*
*i*

*i*
*i*

*i*

*i*
*i*

*i*
*i*

*i*

*i*
*i*

*i*
*i*

*num* *num*

*num* *num*

*num* *num*

*num*

*size* *num*

*size* *num*

*num*

*c*

*c*

**Example:**

**Example:**

**• Amortized cost = 3**

*size* _{i} */ 2* = *size* _{i} _{−} _{1} = *num* _{i} − = *1 16*

_{i}

_{i}

_{1}

_{i}

### 3

### 16 2

### 17

### ) 16 16

### 2 ( ))

### 2 17

### 2 ( 17

### 2 ( 17

### ) 16 16

### 2 ( )

### 32 17

### 2 ( 17

### ˆ

_{1}

### =

### − +

### =

### −

### ⋅

### −

### −

### ⋅

### −

### ⋅ +

### =

### −

### ⋅

### −

### −

### ⋅ +

### =

### Φ

### − Φ

### +

### =

_{i}

_{i}

_{i}_{−}

*i*

*c*

*c*

**Table expansion and ** **contraction**

**• To implement a T** ^{ABLE} **-D** **ELETE**

^{ABLE}

**operation, it is desirable to contract the **

**table when the load factor of the table **

**becomes too small, so that the waste **

**space is not exorbitant.**

**Goal: **

### • **The load factor of the dynamic table is ** **bounded below by a constant.**

### • **The amortized cost of a table operation ** **is bounded above by a constant.**

### • **Set load factor** ≥ *1*

*2*

• **The first n/2 operations are inserted: **

**The second n/2 operations, we perform **

**I, D, D, I, I, D, D, …**

• **Total cost of these n operations is . **
**Hence the amortized cost is .**

• **Set load factor** **(as T****ABLE****_D****ELETE****)**
**(after the contraction, the load factor **

**become )**

*Θ(n*^{2}*)*

*Θ( )*

*n*

≥ *1*
*4*
*1*

*2*

)
Θ*(n*

### ⎪ ⎩

### ⎪ ⎨

### ⎧

### <

### −

### ≥

### = − Φ

### 2 ) 1

### ( ]

### 2 [

### ] [

### 2 ) 1

### ( ]

### [ ]

### [ 2

### ) (

*T* *if*

*T* *T* *num*

*size*

*T* *if*

*T* *size* *T*

*num*

*T* α

### α

**• Initial **

### 0 1

### 0 0

0 0

0 0

### = Φ

### =

### =

### =

### α

*size*

*num*

**TABLE-INSERT**

### • **if , same as before. **

### • **if ** **if**

α_{i−}* _{1}* ≥

*1*

*2*α

_{i}_{−}

*<*

_{1}*1*

*2*

### α

_{i}### <

*1*

*2*

$

*(* *) (* *)*

*(* *) (* *(* *))*

*c* *c*

*size* *num* *size*

*num*
*size* *num* *size*

*num*

*i* *i* *i* *i*

*i* *i* *i*

*i*

*i* *i* *i*

*i*

= + −

= + − − −

= + − − − −

−

− −

Φ Φ _{1}

*1* *1*

*1* *2* *2*

*1* *1*

**Example:**

**Example:**

*size* _{i} = *size* _{i} _{−1} = *16* *,* *num* _{i} = *6*

_{i}

_{i}

_{−1}

_{i}

### $ *(* *) (* *)*

*c* _{i} = *c* _{i} + _{i} − _{i} = + − − −

_{i}

_{i}

_{i}

_{i}

### =

### Φ Φ − _{1} *1* *16*

_{1}

*2* *6* *16*

*2* *5*

*0*

**• If ** ^{α}

_{i}^{≥} ^{1}

^{1}

*2*

2 3 3 2

3

2 3 3 3

2 3 3 3

)) 2 (

( ) )

1 (

2 ( 1

2 ) (

) 2

( 1 ˆ

1 1

1 1

1

1 1

1 1

1 1

1 1

1

+

−

<

+

−

=

+

−

=

−

−

− +

+

=

−

−

− +

=

Φ

− Φ +

=

−

−

−

−

−

−

−

−

−

−

−

−

−

−

*i*
*i*

*i*
*i*

*i*

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*i*
*i*

*size*
*size*

*size*
*size*

*size*
*num*

*size* *num*
*size*

*num*

*size* *num*
*size*

*num*
*c*

*c*

α

**Example:**

**Example:**

*size* _{i} = *size* _{i} _{−1} = *16* *,* *num* _{i} = *8*

_{i}

_{i}

_{−1}

_{i}

### $

*(* *) (* *)*

*c* _{i} = *c* _{i} + _{i} − _{i}

_{i}

_{i}

_{i}

_{i}

### = + ⋅ − − −

### ≤

### Φ Φ − _{1}

_{1}

*1* *2 8 16* *16*

*2* *7* *3*

**Amortized cost of Table insert is O(1).**

**Amortized cost of Table insert is O(1).**

**T** **ABLE** **-D** **ELETE**

**• If **

**does not cause a contraction (i.e., **

### α

_{i−}

_{1}### <

*1*

*2*

### α _{i}

_{i}

*size*

_{i}### = *size*

_{i}_{−1}

_{−1}

$

*(* *) (* *)*

*(* *) (* *(* *))*

*c* *c*

*size* *num* *size*

*num*
*size* *num* *size*

*num*

*i* *i* *i* *i*

*i* *i* *i*

*i*

*i* *i* *i*

*i*

= + −

= + − − −

= + − − − +

−

− −

Φ Φ _{1}

*1* *1*

*1* *2* *2*

*1* *2* *2* *1*

**Example:**

**Example:**

*size* _{i} = *size* _{i} _{−1} = *16* *,* *num* _{i} = *6*

_{i}

_{i}

_{−1}

_{i}

### $ *(* *) (* *)*

*c* _{i} = *c* _{i} + _{i} − _{i} = + − − −

_{i}

_{i}

_{i}

_{i}

### =

### Φ Φ − _{1} *1* *16*

_{1}

*2* *6* *16*

*2* *7*

*2*

### • α _{i} **causes a contraction **

_{i}

*c* *num* *size* *size*

*num*

*i* *i*

*i* *i*

*i*

### = +

### = ^{−} = +

*1*

*2* *4* ^{1} *1*

^{1}

*(* actual cost *)*

1

)) 1 (

) 2 2

((

) )

1 ((

) 1 (

2 ) (

2 ) ( ) 1 (

ˆ

1 1

1

=

+

− +

−

− +

+ +

=

−

−

− +

+

=

Φ

− Φ +

=

−

−

−

*i*
*i*

*i*
*i*

*i*

*i*
*i*

*i*
*i*

*i*

*i*
*i*

*i*
*i*

*num*
*num*

*num*
*num*

*num*

*size* *num*
*size* *num*

*num*
*c*
*c*

**Example:**

**Example:**

*size* *size*

*i* *i*

*num*

*2* = *4* ^{−}

^{1}### =

*i*

### + = *1* *4*

### $

*(* *) (* *) (* *)*

*c*

_{i}### = *c*

_{i}### +

_{i}### −

_{i}### = + + − − −

### =

### Φ Φ

−

_{1}*3 1* *8*

*2* *3* *16*

*2* *4*

*1*

### • **if (Exercise 18.4.3)**

### • **Amortized cost O(1).**

**Amortized cost O(1).**

### α

_{i}_{−}

_{1}