• The time required to perform a sequence of data structure operations is averaged over all the operations performed.

Chapter 17

Amortized Analysis

Lee, Hsiu-Hui

Ack: This presentation is based on the lecture slides from Hsu, Lih-Hsing, as well as various materials from the web.

Amortized Analysis

• The time required to perform a sequence of data structure operations is averaged over all the operations performed.

• An amortized analysis guarantees

the average performance of each operation

in the worst case.

Three common techniques

• aggregate analysis

• accounting method

• potential method

17.1 Aggregate Analysis

• For all n, a sequence of n operations takes

worst time T(n) in total. The amortize cost of each operation is .

n n T( )

EX1-Stack operation

PUSH(S, x) / POP(S) / MULTIPOP(S, k)

MULTIPOP(S, k)

1 while not STACK-EMPTY(S) and k ≠ 0

2 do POP(S)

3 k ← k – 1

Action of M ULTIPOP on a stack S

MULTIPOP(S,4) MULTIPOP(S,7)

top → 23 17 6 39 10 47

⎯⎯

top → 10 47

⎯⎯ ⎯⎯

• PUSH(S, x) / POP(S)

Each runs in O(1) time.

Actual running time for a sequence of n PUSH, POP operations is Θ^(n).

• MULTIPOP(S, k)

pops the top k objects of stack

Total cost : min(s, k) s : stack size

• Analyze a sequence of n PUSH, POP, and

MULTIPOP operation on an initially empty stack.

問每一個 operation 在 worst-case 之 amortized cost?

PUSH O(1)

POP O(1)

MULTIPOP O(n) (the stack size is at most n)

Total cost of n operations: O(n

)

• We can get a better bound.

Total cost of any seq of n operations: O(n)

• Each object can be popped at most once for each time it is pushed.

Number of times the POP can be called on a nonempty stack, including calls within MULTIPOP, is at most the number of PUSH operatons, which is at most n.

Total cost of any seq of n operations: O(n)

better bound !

The amortized cost of an operation is

n n O =

EX2-Incrementing a binary counter

A[0] 每 1 次改變一次 A[1] 每 2 次改變一次 A[i] 每 2ⁱ 次改變一次

I NCREMENT (A) 1 i ← 0

2 while i < length[A] and A[i] = 1 3 do A[i] ← 0

4 i ← i + 1

5 if i < length[A]

6 then A[i] ← 1

length[A]=k Cost : Θ(k)

Analysis:

• A seq of n increment operations : O(nk) (k is the word length)

• Amortize Analysis:

The total number of flips in the sequence is

⎣ ⎦

) ) (

is ( operation per

cost amortized

the

1 2 2

1

2 ₀

log

0

n O n O n

n n

i i n

i

i

=

⇒

=

⎥⎦ <

⎢⎣ ⎥

⎢

∑

∑

=

=

17.2 Accounting method

• We assign differing charges to different operations, with some operations charged

more or less than the actual cost. The amount we charge an operation is called its amortized cost.

Some are charged more than actual cost.

Some are charged less.

• (在aggregate method所有operation有相同的

amortized cost)

• If the amortized cost > actual cost, the difference is treated as credit

• Credit can be used later on to help pay for

operations whose amortized cost is less than

their actual cost.

• If we want analysis with amortized costs to show that in the worst cast the average cost per operation is small, the total amortized cost of a sequence of operations must be an upper bound on the total actual cost of the sequence.

• Moreover, as in aggregate analysis, this

relationship must hold for all sequences of

operations.

• If we denote the actual cost of the ith operation by c_i and the amortized cost of the ith operation by , we require

(i.e. total amortized cost ≥ total actual cost)

for all sequence of n operations.

• The total credit stored in the data structure is the difference between the total actual cost, or

• The total credit must be nonnegative at all times.

cˆ

∑

∑

≥ ⁿ

n i n

i

i c

c

1 1

ˆ

∑

∑

− ⁿ

i

i n

i

i c

c

1 1

ˆ

EX1-Stack operation

n 個運作之後, total amortized cost = O(n)

0 min{k,s}

MULTIPOP

0 1

POP

2 1

PUSH

actual

cost amortized cost

EX2-Incrementing a binary counter

• Each time, there is exactly one 0 that is changed into 1.

• The number of 1’s in the counter is never negative!

• Amortized cost is at most 2 = O(1).

• 執行 n 次 Increment 其 total amortized cost = O(n)

0 1→0 1

2 0→1 1

actual cost amortized cost

17.3 Potential method

• Like the accounting method, but think of the credit as potential stored with the

entire data structure.

• Accounting method stores credit with specific objects.

• Potential method stores potential in the data structure as a whole.

• Can release potential to pay for future operations.

• Most flexible of the amortized analysis methods.

z

: initial data structure

z D

: the data structure of the result after applying the i-th operation to the data structure D

.

z C

: actual cost of the i-th operation.

z $c _i : amotized cost of the i-th operation.

z A potential function Φ maps each data structure D _i to a real number

Φ( D _i ) , which is the potential associated with data structure D _i .

z The amortized cost _{$c i} of the i-th

operation with respect to potential Φ

is defined by

.

• If for all i,

then the amortized cost is always an upper bound on actual cost .

• If then the potential increases.

∑

∑

∑

=

= −

=

Φ

− Φ

+

=

Φ

− Φ

+

=

n

i

n i

n

i

i i

i n

i

i

D D

c

D D

c c

1

0 1

1 1

) (

) (

) ) (

) ( ˆ (

Φ( D_i ) ≥ Φ( D₀ )

$c

c

i n

i i

n

= =

∑ ≥ ∑

1 1

Φ( D_i ) ≥ Φ( D_i−1)

EX1-Stack operation

• the number of objects in the stack of the ith operation.

•

•

Φ( D _i ) =

Φ( D ₀ ) = 0

Φ( D _i ) ≥ 0

PUSH

Φ ( D _i ) − Φ ( D _{i −1} ) = ( s + − = 1 ) s 1

$ ( ) ( )

c _i = c _i + Φ D _i − Φ D _{i − 1} = + = 1 1 2

MULTIPOP

k ' = min{ , } k s

Φ ( D _i ) − Φ ( D _{i −1} ) = − k '

$ ( ) ( ) ' '

c _i = c _i + Φ D _i − Φ D _{i − 1} = − = k k 0

POP

• The amortized cost of each of these three operations is O(1).

• the total amortized cost of a sequence of

$c _i = 0

EX2-Incrementing a binary counter

•

the number of 1’s in the counter after the ith operations

• The ith INCREMENT operation resets bits.

•

• The Counter starts at zero Amortized cost = O(1)

Total amortized cost of n operations is O(n)

i

i

b

D = Φ ( )

t

1 )

( = ≤

− + Φ D

b

b

t

i i

i i

i

i

D b t b t

D − Φ ≤ − + − = −

Φ ( ) (

) (

1 )

1

2 )

1 ( )

1 (

) (

) (

ˆ

= c

+ Φ D

− Φ D

≤ t

+ + − t

=

c

Even if the counter does not start at zero:

0 0

1

0 1

1

2 2

) (

) ˆ (

b b

n b

b

D D

c c

n n

n i

n n

i

i n

i

i

+

−

= +

−

≤

Φ +

Φ

−

=

∑

∑

∑

=

=

=

) (

0

n O k

k b

, as

long as

, since

that particular

in Note

=

≤

17.4 Dynamic tables

• A nice use of amortized analysis.

• Scenario

• Have a table—maybe a hash table.

• Don’t know in advance how many objects will be stored in it.

• When it fills, must reallocate with a larger size, copying all objects into the new larger table.

• When it gets sufficiently small, might want to reallocate with a smaller size.

• Details of table organization not important.

• Goals

• 1. O(1) amortized time per operation.

• 2. Unused space always ≤ constant fraction of allocated space.

• Load factor α = num / size,

• num = # items stored

• size = allocated size.

• If size = 0, then num = 0. Call α = 1.

• Never allow α > 1.

• Keep α > a constant fraction ⇒ goal (2).

Table expansion

• Consider only TABLE-INSERT

• TABLE-DELETE (discuss later)

• Guarantees that load-factor

• (load factor)

• Each time we actually insert an item into the table, it’s an elementary insertion.

2 ) 1

( ≥ α T

α ( ) [ ] T num T [ ]

size T

=

TABLE_INSERT(T, x) 1 if size[T] = 0

2 then allocate table[T] with 1 slot 3 size[T] ← 1

4 if num[T] = size[T]

5 then allocate new-table with 2 ⋅ size[T] slots 6 insert all items in table[T] in new-table 7 free table[T]

8 table[T] ← new-table 9 size[T] ← 2 ⋅ size[T]

10 insert x into table[T]

11 num[T] ← num[T] + 1

Analysis

• Running time: Charge 1 per elementary

insertion. Count only elementary insertions, since all other costs together are constant per call.

• c

= actual cost of ith operation

• If not full, c_i = 1.

• If full, have i - 1 items in the table at the start of the ith operation. Have to copy all i - 1 existing items, then insert ith item⇒c_i = i .

• n operations ⇒c

= O(n)⇒ O(n

) time for n

operations. (?)

Aggregate method:

• amortized cost = 3

⎩ ⎨

= ⎧

otherwise 1

2 of power exact

an is

1 if i- c

i

⎣ ⎦

∑ ∑

=ⁿ

= +

< + =

i

n j

j

i

n n n n

c

1

lg

1

2 2 3

Accounting method:

Each item pays for 3 elementary insertions;

1. inserting itself in the current table,

2. moving itself when the table is expanded, and

3. moving another item that has already been moved once when the table is expanded.

Charge $3 per insertion of x.

$1 pays for x’s insertion.

$1 pays for x to be moved in the future.

$1 pays for some other item to be moved.

Potential method:

• (not expansion)

] [

] [

2 )

( T = ⋅ num T − size T Φ

size _i = size _{i −1}

3

) )

1 (

2 ( )

2 ( 1

) 2

( )

2 ( 1

ˆ

1 1

1

=

−

−

−

−

⋅ +

=

−

⋅

−

−

⋅ +

=

Φ

− Φ +

=

−

−

−

i i

i i

i i

i i

i i

i i

size num

size num

size num

size num

c

c

Example:

size _i = size _{i −1} = 16 , num _i = 13

3

) 16 12

2 ( )

16 13

2 ( 1

) 16 12

2 ( )

16 13

2 ( 1

ˆ

=

−

⋅

−

−

⋅ +

=

−

⋅

−

−

⋅ +

=

Φ

− Φ

+

=

i

c

c

• ^size

^{/ 2 = size}

^{= num}

^{− 1} (expansion)

3

) 1 (

2

)) 1 (

) 1 (

2 (

)) 2 2

( 2

(

) 2

( ) 2

( ˆ

1 1

1

=

−

− +

=

−

−

−

−

−

⋅

−

⋅ +

=

−

⋅

−

−

⋅ +

=

Φ

− Φ +

=

−

−

−

i i

i i

i i

i

i i

i i

i

i i

i i

num num

num num

num num

num

size num

size num

num

c

c

Example:

• Amortized cost = 3

size _i / 2 = size _{i − 1} = num _i − = 1 16

3

16 2

17

) 16 16

2 ( ))

2 17

2 ( 17

2 ( 17

) 16 16

2 ( )

32 17

2 ( 17

ˆ

=

− +

=

−

⋅

−

−

⋅

−

⋅ +

=

−

⋅

−

−

⋅ +

=

Φ

− Φ

+

=

i

c

c

Table expansion and contraction

• To implement a T ^ABLE -D ELETE

operation, it is desirable to contract the

table when the load factor of the table

becomes too small, so that the waste

space is not exorbitant.

Goal:

• The load factor of the dynamic table is bounded below by a constant.

• The amortized cost of a table operation is bounded above by a constant.

• Set load factor ≥ 1

2

• The first n/2 operations are inserted:

The second n/2 operations, we perform

I, D, D, I, I, D, D, …

• Total cost of these n operations is . Hence the amortized cost is .

• Set load factor (as TABLE_DELETE) (after the contraction, the load factor

become )

Θ(n² )

Θ( )

≥ 1 4 1

2

) Θ(n

⎪ ⎩

⎪ ⎨

⎧

<

−

≥

= − Φ

2 ) 1

( ]

2 [

] [

2 ) 1

( ]

[ ]

[ 2

) (

T if

T T num

size

T if

T size T

num

T α

α

• Initial

0 1

0 0

0 0

0 0

= Φ

=

=

=

α

size

num

TABLE-INSERT

• if , same as before.

• if if

α_i−1 ≥ 1 2 α_i−1 < 1 2

α

<

$

( ) ( )

( ) ( ( ))

c c

size num size

num size num size

num

i i i i

i i i

i

i i i

i

= + −

= + − − −

= + − − − −

−

− −

Φ Φ ₁

1 1

1 2 2

1 1

Example:

size _i = size _{i −1} = 16 , num _i = 6

$ ( ) ( )

c _i = c _i + _i − _i = + − − −

=

Φ Φ − ₁ 1 16

2 6 16

2 5

0

• If ^α

^{≥ 1}

2

2 3 3 2

3

2 3 3 3

2 3 3 3

)) 2 (

( ) )

1 (

2 ( 1

2 ) (

) 2

( 1 ˆ

1 1

1 1

1

1 1

1 1

1 1

1 1

1

+

−

<

+

−

=

+

−

=

−

−

− +

+

=

−

−

− +

=

Φ

− Φ +

=

−

−

−

−

−

−

−

−

−

−

−

−

−

−

i i

i i

i

i i

i i

i i

i i

i i

i i

i i

size size

size size

size num

size num size

num

size num size

num c

c

α

Example:

size _i = size _{i −1} = 16 , num _i = 8

$

( ) ( )

c _i = c _i + _i − _i

= + ⋅ − − −

≤

Φ Φ − ₁

1 2 8 16 16

2 7 3

Amortized cost of Table insert is O(1).

Amortized cost of Table insert is O(1).

T ABLE -D ELETE

• If

does not cause a contraction (i.e.,

α

<

α _i

size

= size

₋₁

$

( ) ( )

( ) ( ( ))

c c

size num size

num size num size

num

i i i i

i i i

i

i i i

i

= + −

= + − − −

= + − − − +

−

− −

Φ Φ ₁

1 1

1 2 2

1 2 2 1

Example:

size _i = size _{i −1} = 16 , num _i = 6

$ ( ) ( )

c _i = c _i + _i − _i = + − − −

=

Φ Φ − ₁ 1 16

2 6 16

2 7

2

• α _i causes a contraction

c num size size

num

i i

i i

i

= +

= ⁻ = +

1

2 4 ¹ 1

( actual cost )

1

)) 1 (

) 2 2

((

) )

1 ((

) 1 (

2 ) (

2 ) ( ) 1 (

ˆ

1 1

1

=

+

− +

−

− +

+ +

=

−

−

− +

+

=

Φ

− Φ +

=

−

−

−

i i

i i

i

i i

i i

i

i i

i i

num num

num num

num

size num size num

num c c

Example:

size size

i i

num

2 = 4 ⁻

=

+ = 1 4

$

( ) ( ) ( )

c

= c

+

−

= + + − − −

=

Φ Φ

3 1 8

2 3 16

2 4

1

• if (Exercise 18.4.3)

• Amortized cost O(1).

α

₋

≥ 1

2