Dynamic Programming

Chapter 15

Dynamic Programming

Lee, Hsiu-Hui

Ack: This presentation is based on the lecture slides from Hsu, Lih-Hsing, as

Introduction

• • Dynamic programming Dynamic programming is typically applied to optimization problems.

• In such problem there can be many

solutions. Each solution has a value, and we wish to find a solution with the

optimal value.

The development of a dynamic programming algorithm can be broken into a sequence of four steps:

1. Characterize the structure of an optimal solution.

2. Recursively define the value of an optimal solution.

3. Compute the value of an optimal solution in a bottom up fashion.

4. Construct an optimal solution from computed information.

Assembly-line scheduling

a manufacturing problem

to find the fast way through a factory

a : the assembly time required at station S

An instance of the assembly-line problem with costs

Step 1: The structure of the

fastest way through the factory

• Optimal substructure

An optimal solution to a problem (fastest way through S1, j ) contains within it an optimal solution to subproblems.

(fastest way through S1, j−1 or S 2, j−1).

• Use optimal substructure to construct optimal solution to problem from optimal solutions to subproblems.

To solve problems of finding a fastest way through S1, j and S2, j , solve subproblems of finding a fastest way through S and S

Step 2: A recursive solution

) ]

[ ,

] [

min(

*

f n x f n x

f = + +

2 if

, 1 if

2 if

, 1 if

) ]

1 [

, ]

1 [

] min(

[

) ]

1 [

, ]

1 [

] min(

[

, 2 1

, 1 1

, 2 2

1 , 2 2

2

, 1 1

, 2 2

, 1 1

1 , 1 1

1

≥

=

≥

=

⎪⎩

⎪⎨

⎧

+ +

− +

−

= +

⎪⎩

⎪⎨

⎧

+ +

− +

−

= +

−

−

j j j j

a t

j f a

j f a j e

f

a t

j f a

j f a j e

f

j j

j

j j

j

f_i[j]: the fastest possible time to get a chassis from the starting point through station S_{i, j}

f*: the fastest time to get a chassis all the way through the factory.

• li [ j ] = line # (1 or 2) whose station j − 1 is used in fastest way through Si, j .

• Sli [ j ], j−1 precedes Si, j .

• Defined for i = 1,2 and j = 2, . . . , n.

• l^∗ = line # whose station n is used.

step 3: computing an optimal solution

• Let r_i(j)be the number of references made to f_i[j] in a recursive algorithm.

r₁(n)=r₂(n)=1

r₁(j) = r₂(j)=r₁(j+1)+r₂(j+1)

• The total number of references to all f_i[j] values is Θ(2ⁿ).

• We can do much better if we compute the f_i[j] values in different order from the recursive way. Observe that for j ≥ 2, each value of f_i[j] depends only on the values of f₁[j-1] and f₂[j-1].

F ASTEST -W AY procedure

FASTEST-WAY(a, t, e, x, n) 1 f₁[1] ← e₁ + a_1,1

2 f₂[1] ← e₂ + a_2,1 3 for j ← 2 to n

4 do if f₁[j-1] + a_1,j ≤ f₂[j-1] + t_2,j-1 +a_1,j 5 then f₁[j] ← f₁[j-1] + a_1,j

6 l₁[j] ← 1

7 else f₁[j] ← f₂[j-1] + t_2,j-1 +a_1,j

8 l [j] ← 2

10 then f₂[j] ← f₂[j – 1] + a_2,j

11 l2[j] ← 2

12 else f₂[j] ← f₁[j – 1] + t_1,j-1 + a₂,j

13 l₂[j] ← 1

14 if f₁[n] + x₁ ≤ f₂[n] + x₂ 15 then f^* = f₁[n] + x₁

16 l^* = 1

17 else f^* = f₂[n] + x₂

18 l^* = 2

step 4: constructing the

fastest way through the factory

PRINT-STATIONS(l, l*, n) 1 i ← l*

2 print “line” i “,station” n 3 for j ← n downto 2

4 do i ← l_i[j]

5 print “line” i “,station” j – 1

output

line 1, station 6 line 2, station 5 line 2, station 4 line 1, station 3 line 2, station 2 line 1, station 1

15.2 Matrix-chain multiplication

• A product of matrices is fully

parenthesized if it is either a single matrix, or a product of two fully parenthesized

matrix product, surrounded by

parentheses.

• How to compute where is a matrix for every i.

• Example:

A A _{1 2} ... A _n A _i

A A A A

( ( ( ))) ( (( ) ))

(( )( )) (( ( )) )

((( ) ) )

A A A A A A A A

A A A A A A A A

A A A A

1 2 3 4 1 2 3 4

1 2 3 4 1 2 3 4

1 2 3 4

MATRIX MULTIPLY

MATRIX MULTIPLY(A, B) 1 if columns[A] column[B]

2 then error “incompatible dimensions”

3 else for to rows[A]

4 do for to columns[B]

5 do

6 for to columns[A]

7 do

8 return C

≠

i ←1 j ←1

c i j [ , ] ← 0 k ←1

c i j [ , ] ← c i j [ , ] + A i k B k j [ , ] [ , ]

Complexity:

Let A be a matrix B be a matrix.

Then the complexity is

p × q

q × r

p × × q r

Example:

is a matrix, is a matrix, and is a matrix

Then

takes time.

However

takes time.

A ₁ ^{10 100 ×}

^{100 5 ×}

A₃

5 50 × (( A A _{1 2} ) A ₃ )

10 100 × × + 5 10 × × 5 50 = 7500

( A ₁ ( A A _{2 3} ))

100 × × 5 50 + 10 × 100 × 50 = 75000

The matrix-chain

multiplication problem:

• Given a chain of n matrices, where for i=0,1,…,n, matrix A

has

dimension p

× p

, fully parenthesize the product in a way that

minimizes the number of scalar multiplications.

〉

〈 A

, A

,..., A

A A _{1 2} ... A _n

Counting the number of parenthesizations:

• [Catalan number]

⎪⎩

⎪ ⎨

⎧

≥

−

=

= ∑

=

2 )

( ) (

1 1

)

(

1

n if k

n P k P

n if n

P

k

P n ( ) = C n ( −1 )

= +

⎛

⎝⎜

⎞

⎠⎟ = 1

1

2 4

n

n

n n

Ω(

)

/

Step 1: The structure of an optimal parenthesization

(( A A _{1 2} ... A _k )( A _{k + 1} A _{k + 2} ... A _n ))

Optimal

Combine

Step 2: A recursive solution

• Define m[i, j]= minimum number of

scalar multiplications needed to compute the matrix

• goal m[1, n]

•

A

= A A

.. A

m i j [ , ] =

⎩ ⎨

⎧

≠ +

+ +

=

−

<

≤

m i k m k j p p p i j

j i

j k i

j k

i

{ [ , ] [ 1 , ] }

min

0

1

Step 3: Computing the

optimal costs

MATRIX_CHAIN_ORDER

MATRIX_CHAIN_ORDER(p) 1 n ← length[p] –1

2 for i ← 1 to n 3 do m[i, i] ← 0 4 for l ← 2 to n

5 do for i ← 1 to n – l + 1

6 do j ← i + l – 1

7 m[i, j] ← ∞

8 for k ← i to j – 1

9 do q ← m[i, k] + m[k+1, j]+ p_i-1p_kp_j

10 if q < m[i, j]

11 then m[i, j] ← q

12 s[i, j] ← k

13 return m and s

Complexity: O n ( ³ )

Example:

5 4

5

4 3

4

3 2

3

2 1

2

1 0

1

20 10

10 5

5 15

15 35

35 30

p p

A

p p

A

p p

A

p p

A

p p

A

×

=

×

×

=

×

×

=

×

×

=

×

×

=

×

the m and s table computed by

MATRIX-CHAIN-ORDER for n=6

m[2,5]=

min{

m[2,2]+m[3,5]+p₁p₂p₅=0+2500+35×15×20=13000, m[2,3]+m[4,5]+p₁p₃p₅=2625+1000+35×5×20=7125, m[2,4]+m[5,5]+p₁p₄p₅=4375+0+35×10×20=11374 }

=7125

Step 4: Constructing an

optimal solution

PRINT_OPTIMAL_PARENS(s, i, j) 1 if j = i

2 then print “A”

3 else print “(“

4 PRINT_OPTIMAL_PARENS(s, i, s[i,j])

5 PRINT_OPTIMAL_PARENS(s, s[i,j]+1, j)

6 print “)“

example:

[1,6]

[4,6]

[1,3]

[4,5]

A

A

A

A

[6,6]

A

[1,1]

A

[2,3]

[2,2] [3,3] [4,4] [5,5]

(( A A A₁( _{2 3}))(( A A_{4 5} )A₆ ))

16.3 Elements of dynamic

programming

Optimal substructure

• We say that a problem exhibits optimal substructure if an optimal solution to the problem contains within its optimal

solution to subproblems.

• Example: Matrix-multiplication

problem

1. You show that a solution to the problem consists of making a choice. Making this choice leaves one or more subproblems to be solved.

2. You suppose that for a given problem, you are given the choice that leads to an optimal solution.

3. Given this choice, you determine which subproblems ensue and how to best characterize the resulting space of subproblems.

4. You show that the solutions to the subproblems used

Optimal substructure varies across problem domains in two ways:

1. how many subproblems are used in an optimal solutiion to the original problem, and

2. how many choices we have in determining

which subproblem(s) to use in an optimal

solution.

Overlapping subproblems:

example:

MAXTRIX_CHAIN_ORDER

RECURSIVE_MATRIX_CHAIN

RECURSIVE_MATRIX_CHAIN(p, i, j) 1 if i = j

2 then return 0 3 m[i, j] ← ∞

4 for k ← i to j – 1

5 do q ← RMC(p,i,k) + RMC(p,k+1,j) + p

p

p

6 if q < m[i, j]

7 then m[i, j] ← q

8 return m[i, j]

The recursion tree for the computation

of RECURSUVE-MATRIX-CHAIN(P, 1, 4)

• We can prove that T(n) =Ω(2

) using substitution method.

⎪⎩

⎪⎨

⎧

>

+

− +

+

≥

≥

∑

= 1

1

1 )

1 ) (

) ( ( 1

) (

1 ) 1 (

n

k

n for k

n T k

T n

T

T

T n T i n

i n

( ) ≥ ∑ ( ) +

=

−

2

1 1

Solution:

1. bottom up

2. memorization (memorize the natural, but

1 1

2 0 1

1

1 0

2 )

2 2

( )

1 2

( 2

2 2

2 2

) (

2 1

) 1 (

−

−

−

=

−

=

−

≥ +

−

= +

−

=

+

= +

≥

=

≥

∑

∑

n n

n

n i n i

i

i

n n

n n

n

T

T

MEMORIZED_MATRIX_CHAIN

MEMORIZED_MATRIX_CHAIN(p) 1 n ← length[p] –1

2 for i ← 1 to n

3 do for j ← 1 to n 4 do m[i, j] ← ∞

5 return LOOKUP_CHAIN (p,1,n)

LOOKUP_CHAIN

LOOKUP_CHAIN(p, i, j) 1 if m[i, j] < ∞

2 then return m[i, j]

3 if i = j

4 then m[i, j] ← 0

5 else for k ← i to j – 1

6 do q ← LC(p, i, k) +LC(p, k+1, j)+p_i-1p_kp_j 7 if q < m[i, j]

8 then m[i, j] ← q

16.4 Longest Common Subsequence

X = < A, B, C, B, D, A, B >

Y = < B, D, C, A, B, A >

• < B, C, A > is a common subsequence of both X and Y.

• < B, C, B, A > or < B, C, A, B > is the

longest common subsequence of X and Y.

Longest-common-

subsequence problem:

• Given two sequences

X = <x₁,x₂,...,x_m> Y = <y₁,y₂,...,y_n>

• To find a maximum length common subsequence (LCS) of X and Y.

• Define X

: ^{the ith prefix of X}

X_i = < x₁,x₂,...,x_i >

e.g. X=<A,B,C,B,D,A,B>

Theorem 15.1

(Optimal substructure of LCS)

Let X = <x

,x

,...,x

> and Y = <y

,y

,...,y

>

and let Z = <z

,z

,...,z

> be any LCS of X and Y.

1. If x_m = y_n

then z_k = x_m = y_n and Z_k-1 is an LCS of X_m-1 and Y_n-1. 2. If x_m ≠ y_n

then z_k ≠ x_m implies Z is an LCS of X_m-1 and Y.

3. If x_m ≠ y_n

then z_k ≠ y_n implies Z is an LCS of X and Y_n-1.

A recursive solution to subproblem

• Define c [i, j] is the length of the LCS of X

and Y

.

⎪ ⎩

⎪ ⎨

⎧

≠

−

−

+

−

−

=

j i

j i

y x

>

i,j j

i c j

i c

=y x

>

i,j j

i c

= j

= i j

i c

and if

and if

or if

0 }

] , 1 [

], 1 ,

[ max{

0 1

] 1 ,

1 [

0 0

0 ]

,

[

Computing the length of an LCS

LCS_LENGTH(X,Y) 1 m ← length[X]

2 n ← length[Y]

3 for i ← 1 to m

4 do c[i, 0] ← 0 5 for j ← 1 to n 6 do c[0, j] ← 0 7 for i ← 1 to m

8 do for j ← 1 to n

9 do if x

= y

10 then c[i, j] ← c[i-1, j-1]+1 11 b[i, j] ← “Ñ”

12 else if c[i–1, j] ≥ c[i, j-1]

13 then c[i, j] ← c[i-1, j]

14 b[i, j] ← “Ï”

15 else c[i, j] ← c[i, j-1]

Complexity: O(mn)

PRINT_LCS

PRINT_LCS(b, X, i, j ) 1 if i = 0 or j = 0

2 then return 3 if b[i, j] = “Ñ”

4 then PRINT_LCS(b, X, i-1, j-1)

5 print x

6 elseif b[i, j] = “Ï”

Complexity: O(

Complexity: O( m+n m+n ) )

15.5 Optimal Binary search trees

cost:2.80 cost:2.75

optimal!!

expected cost

• the expected cost of a search in T is

∑ ∑

= =

=

n +

i

n

i

i

i q

p

1 0

1

∑ ∑

∑ ∑

= =

⋅ +

⋅ +

=

⋅ + +

⋅ +

=

n n

i n

i

n

i

i T i

i T

q d

p k

q d

p k

1 0

) ( depth )

( depth 1

) 1 ) ( depth (

) 1 ) ( depth (

T]

in cost E[search

• For a given set of probabilities, our goal is to construct a binary search tree whose expected search is smallest. We call such a tree an

optimal binary search tree.

Step 1: The structure of an optimal binary search tree

• Consider any subtree of a binary search tree. It must contain keys in a contiguous range k

, ...,k

, for some 1 ≤ i ≤ j ≤ n. In addition, a subtree that contains keys k

, ..., k

must also have as its leaves the dummy keys d

, ..., d

.

• If an optimal binary search tree T has a subtree T'

containing keys k

, ..., k

, then this subtree T' must

be optimal as well for the subproblem with keys

Step 2: A recursive solution

⎪⎩

⎪⎨

⎧

≤

−

= +

+ +

= −

+

=

≤

≤

−

−

=

=

∑

∑

. if

, 1 if

)}

, ( ]

, 1 [

] 1 ,

[ { ] min

, [

) , (

1 1

j i

i j j

i w j

r e r

i e

q j

i e

q p

j i w

j r i

i j

i l

l j

i l

l

e[i,j]: the expected cost of searching an optimal binary search tree containing the keys ki, … kj

Step 3:computing the expected search cost of an OBST

O PTIMAL -BST(p,q,n)

1 for i ← 1 to n + 1 2 do e[i, i – 1] ← q

3 w[i, i – 1] ← q

4 for l ← 1 to n

5 do for i ← 1 to n – l + 1 6 do j ← i + l – 1

e[i, j] ← ∞

9 for r ← i to j

10 do t ← e[i, r –1]+e[r +1, j]+w[i, j]

11 if t < e[i, j]

12 then e[i, j] ← t

13 root[i, j] ← r

14 return e and root

OPTIMAL-BST procedure takes Θ(n³), just like MATRIX-CHAIN-ORDER

The table e[i,j], w[i,j], and

root[i,j] computer by O

-BST on

the key distribution.

• Knuth has shown that there are always roots of optimal subtrees such that root[i, j –1] ≤

root[i+1, j] for all 1 ≤ i ≤ j ≤ n.

• We can use this fact to modify Optimal-BST

procedure to run in Θ(n

) time.