Dynamic programming has been applied to numerous areas in mathematics, science, engineering, business, medicine, information systems, bioinformatics, among others. There are few disciplines in which there are no optimization problems to which DP has been applied. In this book, we focus on applications to operations research and computer science. Most of these applications can be classified into a few major groups: applications involving graph routing, sequencing, selection, partitioning, production and distribution, divide-and-conquer, probabilistic or weighted transitions, string processing, and nonop-timization problems. These groups are neither disjoint nor complete; some applications fit in more than one group, and others fit in none of them. (In the following, the parenthesized labels identify specific problems we discuss in this Chapter.)
1. Graph routing problems are associated with finding shortest (or longest) paths in graphs. The problem is especially simple when the graph is acyclic (SPA, Sect. 2.43). Special cases include graphs where the nodes are grouped in stages (SCP, Sect. 2.38) or in separate lines (ASMBAL, Sect.
2.4). A more complicated problem allows the graph to be cyclic (SPC, Sect. 2.44), and even have negative branch labels. For cyclic graphs, we may wish to find longest simple paths (LSP, Sect. 2.26) or shortest Hamil-tonian paths (TSP, Sect. 2.47). For some applications, neither the source nor the target are specified; instead, we are to find the “all-pairs shortest-paths” (APSP, Sect. 2.2) from each node p to each other node q, for all pairs (p, q). The problem of finding a path whose maximal branch is minimal (MINMAX, Sect. 2.28) is of special interest since it involves non-additive costs. Many other classes of problems can be formulated as graph routing problems whose branch distances must generally be derived rather than being explicitly given in a distance array.
2. Sequencing problems are associated with finding the optimal ordering or permutation of a set of objects. The set of objects may be data to be placed in an array for linear searching (LINSRC, Sect. 2.24), a set of program
A. Lew and H. Mauch: Applications of Dynamic Programming, Studies in Computational Intel-ligence (SCI) 38, 45–100 (2007)
www.springerlink.com Springer-Verlag Berlin Heidelberg 2007c
files to be placed on a tape (PERM, Sect. 2.32), or a set of processes to be scheduled. In scheduling problems, processes may be jobs to be scheduled for execution on a CPU to minimize average turnaround time (SPT, Sect. 2.45), or multitask jobs to be scheduled in flowshop fashion (FLOWSHOP, Sect. 2.12), or disk file access requests to be scheduled to minimize total seek times (SEEK, Sect. 2.39). Sequencing problems can also viewed as a class of assignment problems where each object is assigned to some position in the sequence (ASSIGN, Sect. 2.5).
3. Selection problems are associated with finding optimal (usually proper) subsets of a given set of objects. One example is the problem of finding a spanning tree of a graph that has minimum weight (MWST, Sect. 2.29);
this tree consists of a subset of branches with minimum total weight that spans the graph while satisfying the eligibility constraint of acyclicness.
In knapsack problems, objects have values and weights, and the objective is to select a subset with optimal total value such that the total weight does not exceed the size of the knapsack, where there may be at most one object of a given type (KS01, Sect. 2.20) or multiple objects of a given type (KSINT, Sect. 2.22 and RDP, Sect. 2.36). In scheduling problems, a subset of jobs to be executed is selected taking into account eligibility constraints based upon, for example, deadlines (DEADLINE, Sect. 2.8) or start and stop times (INTVL, Sect. 2.16). Integer linear programming problems can be solved using the same DP approach adopted for knapsack problems (ILP, Sect. 2.14), and in turn knapsack problems can be solved by formulating them as integer linear programming problems (ILPKNAP, Sect. 2.15). Some covering problems can also be formulated as a variation of a knapsack problem (COV, Sect. 2.7; KSCOV, Sect. 2.21).
4. Partitioning or clustering problems are characterized by having a set S of objects of size |S| = N that are to be grouped into disjoint subsets, called a partition, in an optimal fashion. The rank K of the partition, i.e., the number of disjoint subsets or clusters (1≤ K ≤ N), either may be given in advance or may be left to be optimally determined. A sequential partitioning problem, also known as a segmentation problem, is one in which S is considered an ordered sequence of objects, and nonadjacent objects can only be in a cluster or segment if all intervening objects are in the same segment. Examples of such segmentation problems include curve fitting (SEGLINE, Sect. 2.41) and program pagination (SEGPAGE, Sect. 2.41). A partitioning problem may also be viewed as that of assigning each object to one of a given number of clusters. In one example of such an assignment problem (ASSIGN, Sect. 2.5), we assume the special case where K = N . It is more common, however, for K to be well less than N . Some covering problems, as mentioned above, can also be formulated as a variation of a partitioning problem.
5. Distribution problems are characterized by having a set of objects that are to be distributed from a set of suppliers to a set of receivers in an opti-mal fashion, where there are specified costs associated with destination y
receiving objects from source x. In transportation problems, this is mod-eled by having a flow from x to y (TRANSPO, Sect. 2.46). In some simpler special cases, there is one source and multiple destinations, and distribu-tion costs depend on the allotments, i.e. the number of objects allocated to each destination (ALLOT, Sect. 2.1; ODP, Sect. 2.31). Another spe-cial case is that of assignment problems where sources must be matched (paired) with destinations (ASSIGN, Sect. 2.5). We also note that many distribution problems can be modeled as integer linear programming prob-lems, and handled as above. Furthermore, it is common for distribution problems to have probabilistic elements as well, so that they must be handled as below.
6. Production or inventory or replacement problems are characterized by having a resource, units of which can be produced (by a supplier) or demanded and consumed (by a receiver) at each of a series of stages, where there are specified costs of producing or not producing additional units of resource to replace consumed ones, or to add to existing inven-tory to meet future demand (PROD, Sect. 2.34; INVENT, Sect. 2.17;
REPLACE, Sect. 2.37; LOT, Sect. 2.25). Investment problems (INVEST, Sect. 2.18; INVESTWLV, Sect. 2.19) can be modeled in like fashion, where the amount that can be invested corresponds to the inventory, and gains and losses from investments correspond to demands.
7. Optimal binary tree problems are associated with situations where a de-cision divides the original problem into two subproblems whose solutions can be solved separately and combined to obtain the solution of the orig-inal problem. This is similar to what are known as divide-and-conquer algorithms. The main characteristic is that each decision leads to multiple next-states rather than a single one. Examples include constructing an optimal binary search tree (BST, Sect. 2.6), constructing an optimal al-phabetic radix or prefix-code tree (ARC, Sect. 2.3), determining the best way to multiply a chain of matrices (MCM, Sect. 2.27), and determining the best way to obtain a set of order statistics (SELECT, Sect. 2.42). The DPFEs for these problems are nonserial (second-order).
8. Probabilistic problems have probabilities associated with next-states. Un-like the aforementioned divide-and-conquer problems, in probabilistic problems each decision results in single next-state determined by chance from a set of alternatives. The DPFEs for these problems (INVEST, Sect. 2.18; INVESTWLV, Sect. 2.19; PROD, Sect. 2.34; PRODRAP, Sect. 2.35) are nonserial. Many probabilistic problems are production or investment problems, where the state is the current inventory or balance, the decision is how much to produce or invest, and the next-states may increase or decrease, sometimes based on random external factors such as demand. We note that there are also serial DP problems where probabil-ities are not associated with next-states, but instead are associated with the reward function (RDP, Sect. 2.36); in the simplest case, the reward is an expected value computed from given probabilities.
9. Probabilistic problems are only one class of problems in which next-states are weighted. Boolean weights are used, for example, for a scheduling prob-lem (INTVL3, Sect. 2.16) and a routing probprob-lem (APSPFW, Sect. 2.2).
Such weights can also be based on “discounting” (DPP, Sect. 2.9).
10. Problems involving string processing are also of interest, especially in the fields of text processing and computational biology. We discuss the prob-lems of “editing” or transforming one string to another (EDP, Sect. 2.10) and of finding “matching” subsequences (LCS, Sect. 2.23).
11. Nonoptimization problems, in the context of sequential decision processes, are those where decisions are not made for the purpose of optimizing some objective function. Instead, decisions are made only to satisfy certain con-straints, especially so as to ensure that a specific base state is reached.
For example, DP can be used to solve recurrence relations, such as for the Fibonacci sequence (FIB, Sect. 2.11) and for the Tower of Hanoi prob-lem (HANOI, Sect. 2.13). In addition, DP can also be used to determine sequences of moves that solve some problem, such as for certain puzzles (HANOI, Sect. 2.13; NIM, Sect. 2.30; POUR, Sect. 2.33).
There are of course other ways to categorize the various applications.
One other way is by discipline. For example, applications covered in a typ-ical operations research textbook are (SCP, ALLOT, ALLOTm, ILP, PRO-DRAP, INVESTWLV) in [21, Chap. 11] or (NIM, POUR, SCP, INVENT, ALLOTf, KSINT, REPLACE, DDP, TSP, MINMAX, ALLOTm, LOT) in [63, Chap. 18]. Applications covered in a typical computer science textbook [10, Chap. 15] include (ASMBAL, MCM, LSP, LCS, BST). In this book, we treat all of these applications, among many others, to demonstrate the gener-ality of our formalism and the utility of our DP software tool for solving DP problems.
In this chapter, we show how each of the problems referred to here can be solved using dynamic programming. Specifically, we formulate a DPFE for each problem. In later chapters, these DPFEs are expressed in the text-based language gDPS, which serves as the input source programs for our DP software tool that translates them into object programs whose execution solves the problems. We may view gDPS both as a mathematical specification language and as a programming language.
It should be noted that a number of the problems included here can be solved much more efficiently by greedy algorithms. In these cases, we present less efficient DP solutions in part to demonstrate the generality of DP as a method to solve optimization problems, and in part to provide a large sample of DP formulations. Having such a large sample for reference, it should be easier to formulate DP solutions to new problems. Furthermore, the set of examples of the use of our software tool is therefore also large, which should make it easier to learn to use this tool. Finally, it should be emphasized that we often can modify DP to solve variations of problems for which greedy algorithms are inapplicable.
The following sections present the problems in alphabetical order of their names.
2.1 Optimal Allotment Problem (ALLOT)
The optimal allotment problem is that of deciding how to distribute a limited amount of resources to a set of users of these resources, where there are spec-ified costs or profits associated with allotting units of the resource to users.
The optimal distribution problem ODP (Sect. 2.31) may be regarded as an allotment problem. The problem may also be regarded as a variation of the knapsack problem KSINT (Sect. 2.22).
Assume there are M total units of the resource, and let C(k, d) be the cost or profit associated with allotting d units to user k, where d = 0, . . . , M and k = 1, . . . , N . Suppose we make the allotment decisions in stages, ini-tially allotting d1units to user 1, then d2 units to user 2, etc. This arbitrary sequencing 1, 2, . . . , N can be assumed since only the quantities allotted mat-ter, not the sequence in which they are made. We define the state (k, m) as remaining number m of units of resource at stage k. The cost of deciding to allot d units at stage k to user k is C(k, d). The next-state is (k + 1, m− d).
The DPFE is
f (k, m) = min
d∈{0,...,m}{C(k, d) + f(k + 1, m − d)}. (2.1) The goal is to find f (1, M ) with base-conditions f (N +1, m) = 0 when m≥ 0.
If we allow d > m (bounding d by M instead), we may use the additional base-condition f (N + 1, m) =∞ when m < 0 to prevent allotting more resources than is available.
For instance, let M = 4, N = 3 and
(Ck,d)k∈{1,2,3};d∈{0,...,4}=
⎛
⎝∞ 1.0 0.8 0.4 0.0
∞ 1.0 0.5 0.0 0.0
∞ 1.0 0.6 0.3 0.0
⎞
⎠ .
Then f (1, M ) = 1.0 + 0.5 + 1.0 = 2.5 for the optimal sequence of allotments d1= 1, d2= 2, d3= 1.
This simple allotment problem can be generalized in several ways.
• In the ALLOTt problem, in [21, Example 3, pp.549–552], allotment deci-sions and their costs are defined in separate tables. (ODP and ALLOTm also have tabular costs.)
• In the ALLOTf problem, in [63, Example 5, pp.975–977], the costs are defined nontabularly, i.e., by general functions.
• In the ALLOTm problem, in [63, Example 14, pp.998–999], costs are mul-tiplicative rather than additive. (ALLOTt also has mulmul-tiplicative costs.)
There are also probabilistic DP problems where allotment costs are random variables; for many such problems, we may simply use the expected values of these costs in nonprobabilistic DPFEs.
2.2 All-Pairs Shortest Paths Problem (APSP)
In Sect. 1.1.11, we discussed the All-Pairs Shortest Paths (APSP) problem of finding the shortest path from any node p to any other node q, where p and q are arbitrary nodes in a set S. We may of course use any general designated-source or target-state shortest path algorithm, such as (SPC, Sect. 2.44), varying p and q repetitively. APSP algorithms can be more efficient since it is possible to reuse calculations when shortest paths are computed in batches rather than individually. Such reuse requires that calculations be suitably ordered. (We do not address this efficiency issue here.) In our implementation, specifically in our generation of the solution tree, subtrees are recalculated.
In the DPFEs given in Sect. 1.1.11, the goal is to compute F (k, p, q) which is regarded as a matrix equation where p and q are row and column subscripts for a matrix F(k). The DPFE gives F(k) in terms of the matrix F(k−1). The relaxation DPFE (1.43) is:
F (k, p, q) = min
r∈S{b(p, r) + F (k − 1, r, q)}, (2.2) for k > 0, with F (0, p, q) = 0 if p = q and F (0, p, q) =∞ if p = q. Recall that we assume here that b(p, p) = 0 for all p.
An alternative DPFE is the Floyd-Warshall DPFE (1.44):
F (k, p, q) = min{F (k − 1, p, q), F (k − 1, p, k) + F (k − 1, k, q)}, (2.3) for k > 0, where k is the highest index of the set of nodes. The base cases are F (0, p, q) = 0 if p = q and F (0, p, q) = b(p, q) if p = q. Both DPFEs define a matrix of values F(k), which is to be determined for one pair of subscripts (p, q) at a time (rather than in a batch).
These two formulations differ is one significant way. In the former case (APSP), the decision space is the set of nodes S. The problem can be solved in much the same fashion as for SPC, except that the target is a parameter rather than fixed. In the latter case (APSPFW), the decision space is Boolean, reflecting whether or not a path shorter than the prior one has been found.
The Floyd-Warshall DPFE can be reformulated using transition weights as follows:
F (k, p, q) = min
d∈{0,1}{(1 − d).F (k − 1, p, q) + d.F (k − 1, p, k) + d.F (k − 1, k, q)}.
(2.4) For example, consider the graph whose adjacency matrix C, whose entries are the branch distances b, is
C =
⎛
⎜⎜
⎝
∞ 3 5 ∞
∞ ∞ 1 8
∞ 2 ∞ 5
∞ ∞ ∞ ∞
⎞
⎟⎟
⎠
where C[p][q] = b(p, q) = ∞ if there is no branch from node p to node q.
This is the same example given in Sect. 1.1.10, and is also used for SPC, as displayed in Fig. 2.8.
For APSP, which uses (2.2), assuming p = 0 and q = 3, we find that F (3, 0, 3) = 9. From the solution obtained during the course of this calculation, we also find that F (2, 1, 3) = 6 and F (1, 2, 3) = 5. However, some values such as for F (3, 0, 2) = 4 must be found separately, such as by assuming p = 0 and q = 2.
For APSPFW, which uses (2.4), assuming p = 0 and q = 3, we find that F (3, 0, 3) = 9. From the solution obtained during the course of this calculation, we also find that F (2, 0, 3) = 9 and F (1, 0, 3) = 11, among many other values.
However, some values such as for F (3, 1, 3) = 6 must be found separately, such as by assuming p = 1 and q = 3.
2.3 Optimal Alphabetic Radix-Code Tree Problem (ARC)
The optimal alphabetic radix code tree problem [23] is a variation of the Huffman code tree problem. The Huffman code tree can be obtained by the
“greedy” heuristic of choosing to combine in a subtree the two nodes having smallest weight, and replacing these two nodes a and b by a new node c having a weight equal to the sum of the weights of nodes a and b. The alphabetic variation constrains each choice to that of combining only adjacent nodes, with each new node being placed in the same position as the nodes it replaces.
Given this constraint, the greedy heuristic is no longer necessarily optimal, but the problem can be solved using DP. Since each tree can be represented as a parenthesized list of its leaves in inorder, the problem is equivalent to finding an optimal parenthesization, and can be solved in a fashion similar to BST and MCM. How costs can be attributed separably to the individual decisions is the key to solving this problem. We note that a Huffman tree may be regarded as constructed from the bottom up, where the first decision is to choose the leaves that should be initially combined. On the other hand, the alphabetic radix code tree will be constructed from the top down, where the first decision is to choose the root of the overall tree, which partitions the leaves into left and right subsets. The main problem is to define separable partitioning costs.
A radix code tree has one useful property that provides the key to solving the optimization problem. The total cost of the tree is the sum of the costs of the internal nodes, and the cost of an internal node is the sum of the leaves
in the subtree rooted at that node. Internal nodes, of course, correspond to decisions. Thus, we let the cost of an internal node of an alphabetic radix code tree equal the sum of the leaves in the subtree rooted at that node. This cost is the same regardless of how the leaves are to be partitioned. We may then solve the constrained problem by using a DPFE of the same top-down form as for MCM. Given S = (w0, w1, . . . , wn−1) is a list (i.e., an ordered sequence) of weights associated with leaves, we define the state to be a pair (i, j), which represents the list (wi, w2, . . . , wj). Then
f (i, j) = min
i≤d<j{c(i, j, d) + f(i, d) + f(d + 1, j)} if i < j, (2.5) where c(i, j, d) =j
k=iwk. The goal is to find f (0, n− 1) with base condition f (i, j) = 0 when i = j.
For example, if S = (1, 2, 3, 4) initially, the optimal tree is ((1, 2), 3), 4) and f (S) = 3+6+10 = 19. From initial state (0, 3) representing the list (1, 2, 3, 4), the initial decision is d = 2 which results in two next-states, state (0, 2) representing the list (1, 2, 3) and state (3, 3) representing the list (4); the cost of this decision is the sum 1+2+3+4 = 10. As a second example, if S = (2, 3, 3, 4) initially, the optimal tree is ((2, 3), (3, 4)) and f (S) = 5 + 7 + 12 = 24.
2.4 Assembly Line Balancing (ASMBAL)
In what are known collectively as assembly line balancing problems, a product can be assembled by going through a series of processing stations. There are usually costs associated with processing step, and other costs associated with going from one station to another. In general, it is desirable to “balance” the workload among the processing stations in a fashion that depends upon how the various costs are defined. We discuss here one simple example of such assembly line balancing problems.
This “scheduling” example, described in [10, pp.324–331], may be regarded as a variation of the shortest path problem for acyclic graphs (SPA, see Sect. 2.43) or the staged version (SCP, see Sect. 2.38). In this problem, the nodes of the graph are grouped into stages and also into lines. Transitions can only be made from a node at stage k and line i to a node at stage k + 1 and line j, at a cost c(k, i, j). Usually, c(k, i, i) = 0, i.e. at any stage k there is a cost associated with switching lines, but not for staying in the same line.
In addition to a transition cost c (associated with branches), there may be another cost v (associated with nodes). An initial state s and terminal state t are also defined; for simplicity, we adopt the convention that s and t are in line 0, as opposed to a special third line. Then the initial decision is made from s to enter line j at a cost c(0, 0, j), and a final decision is made from any line j to enter t at a cost c(N, j, 0).
The graph associated with the assembly line balancing problem is acyclic, so it can be topologically sorted by stages. We number the nodes in such a
topological order from 0 to 13, with 0 as the initial state s, 13 as the terminal state t. Assume the node costs for the 14 nodes are
v = (0, 7, 8, 9, 5, 3, 6, 4, 4, 8, 5, 4, 7, 0),
and the branch costs b are given by the following weighted adjacency matrix:
⎛
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎜⎜
⎝
∞ 2 4 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ 0 2 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ 2 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0 3 ∞ ∞ ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 1 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ 0 1 ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ 2 0 ∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0 3 ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 2 0 ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0 4 ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 1 0 ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 3
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 2
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
⎞
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎟⎟
⎠
where b(i, j) =∞ if there is no branch from node i to node j.
For 1 ≤ i ≤ 12, node i is the ((i + 1)/2)-th node in line 0 if i is odd, whereas node i is the (i/2)-th node in line 1 if i is even. Note that there are N = 6 stages (not counting s and t). Fig. 2.1 shows the assembly line instance under consideration.
7 9 3 4 8 4
8 5 6 4 5 7
2
2
3
1
1
2
3
2
4
1
3
2 2
4
0 0 0 0 0
0 0 0 0 0
target t node 13 source s
node 0
line 0
line 1
stage 2 stage 3 stage 4 stage 5 stage 6
node 3 node 5 node 7 node 9
node 4 node 6 node 8 node 10 node 12
node 2
node 1 node 11
stage 1
Fig. 2.1. Instance of an Assembly Line Balancing Problem
Such assembly line balancing problems can be solved by a slight modifi-cation to the solution to the stagecoach problem (SCP, Sect. 2.38). The main change to SCP is that of adding each node-cost v to all of its outgoing (or al-ternatively all of its incoming) branches. That is, we would define the net cost or reward function by R(k, i, j) = v(k, i) + c(k, i, j). Furthermore, the next-state function is T (k, i, j) = (k + 1, j). The goal is (0, 0). The base-condition is f (k, i) = 0 when k > N . We note that in this staged formulation, only the
non-infinite members of b need to be stored in an array. The successors of each node are constrained to be nodes in the next stage, which can be determined as a function of their subscripts as opposed to being specified in a table.
In the common case where there is a cost of zero for staying in the same line, c(k, i, i) = 0, then only the nonzero costs have to be provided as part of the problem data. A modified reward function R(k, i, j) could test for whether i = j and if so yield a value of 0, instead of looking up its value in a table or array. Thus, this problem can be solved using the staged DPFE
f (k, i) = min
j {R(k, i, j) + f (k + 1, j)}, (2.6) for f (0, 0) given the base condition f (N + 1, i) = 0. For the above example, f (0, 0) = (0 + 2) + (7 + 2) + (5 + 1) + (3 + 1) + (4 + 0) + (5 + 1) + (4 + 3) = 38, for the optimal sequence of line changes 0,1,0,1,1,0, not counting the convention that we start and end in line 0.
The assembly line balancing problem can also be solved as an ordinary shortest path in an acyclic graph (SPA, Sect. 2.43) problem, where the reward function adds node-costs to branch-costs as above. In effect, we can simply ignore stage numbers. For the above example, we would solve the DPFE
f (i) = min
j {v(i) + b(i, j) + f(j)}, (2.7) for f (s) given the base condition f (t) = 0. For the above example, we obtain the goal f (0) = 38. The shortest path from s = 0 to t = 13 goes through the sequence of nodes 0,1,4,5,8,10,11,13, and has length 38.
2.5 Optimal Assignment Problem (ASSIGN)
In an assignment or matching problem, each member of a set B must be uniquely assigned (or “distributed”) to a member of a set A. If A is ordered, then a matching may also be regarded as a permutation of A.
A permutation B = (b0, b1, . . . , bn−1) of A = (a0, a1, . . . , an−1) can be obtained by deciding, for each member biof B which member ajof A to assign to it through a bijection{0, . . . , n − 1} → {0, . . . , n − 1}. These assignment decisions can be made in any order, so we let i be a stage number and let c(i, j) be the cost of assigning aj to bi at stage i. Since we require unique assignments, at each stage we keep track of the members of A that have not yet been assigned. This set S is incorporated into our definition of state.
Specifically, the state is (k, S) where k is a stage number and S is a set. A decision in state (k, S) chooses a member d∈ S, at a cost C(k, S, d) that in general may also be a function of S. The next-state is (k + 1, S− {d}). The DPFE is
f (k, S) = min
d∈S{C(k, S, d) + f(k + 1, S − {d})}.
Within this framework, we can solve a variety of different assignment or distribution problems by using different definitions of the cost function. For example, let C(k, S, d) =
i∈S(wi) + wd. Then the optimal assignment cor-responds to the solution of the SPT problem (see Sect. 2.45). (We note that
i∈S(wi) = TTLWGT−
i∈S(wi), where TTLWGT is the sum of all the weights.) Thus, the optimization problem can be solved using the DPFE
f (k, S) = min
d∈S{
i∈S
(wi) + wd+ f (k + 1, S− {d})}. (2.8)
The base-condition is f (k, S) = 0 when k = n + 1 or S =∅. The goal is find f (1, S∗), where S∗ is the originally given set of N processes.
For instance, if S∗ = {0, 1, 2} with weights (w0, w1, w2) = (3, 5, 2), then f (1, S∗) = 2 + 5 + 10 = 17 for the optimal sequence of assignment decisions d1 = 2, d2 = 0, d3 = 1. Thus, the optimal permutation of A∗ = (3, 5, 2) is B∗= (2, 3, 5).
2.6 Optimal Binary Search Tree Problem (BST)
This problem is described in [10, pp.356–362]. Assume a set of n data items X = {x0, . . . , xn−1} and a total order defined on these items is given. The access probability of a data item xi is p(xi) or pi for short. (Note that
n−1
i=0 pi = 1.)
The task is to build a binary search tree that has minimal cost, where the cost of the tree is defined as
n−1
i=0
(pilevel(xi))
and level(xi) denotes the level (depth) of the node corresponding to data item xiin the tree. Note that items can be stored in internal nodes of the tree, not only in leaves.
We give two alternative approaches to solve this problem using DP. In the first formulation we define the state to be the set S of items to be arranged in the tree. The DP functional equation can be expressed as
f (S) = min
α∈S{f(Sl) + f (Sr) + r(α, S)} if S = ∅
0 if S =∅, (2.9)
where Sl={x ∈ S : x < α} is the remaining set of items that are smaller than the decision α (and thus appear to the left of α) and Sr ={x ∈ S : x > α}
is the remaining set of items that are larger than α (and thus appear to the right of α) and the cost of the decision is defined as
r(α, S) =
x∈S
p(x).
Using an alternative base case the DP functional equation can be expressed as
f (S) = min
α∈S{f(Sl) + f (Sr) + r(α, S)} if |S| > 1
p(x) if S ={x},
The goal is to compute f (X).
A second DP functional equation formulates the problem by defining a state to be a pair of integers providing the start index and the end index of the data items to be arranged (an approach similar to the DP model for the MCM problem in Sect. 2.27.) For this formulation we require, without loss of generality, that the data items X = (x0, . . . , xn−1) are already ordered. Then the DP functional equation can be stated as
f (i, j) =
⎧⎪
⎨
⎪⎩
k∈{i,...,j}min {f(i, k − 1) + f(k + 1, j) +
j l=i
pl} if i ≤ j
0 if i > j.
(2.10)
Using an alternative base case the DP functional equation can be expressed as
f (i, j) =
⎧⎪
⎨
⎪⎩
k∈{i,...,j}min {f(i, k − 1) + f(k + 1, j) +
j l=i
pl} if i < j
pi if i = j.
In this second model the goal is to compute f (0, n− 1).
We consider the following instance of this problem. We have the following 5 data items, listed in their lexicographical order: (A, B, C, D, E). Their re-spective search probabilities are (0.25, 0.05, 0.2, 0.4, 0.1). The optimal value of this instance is f (X) = 1.9 (in terms of the second DP model: f (0, 4) = 1.9), which corresponds to the optimal binary search tree depicted in Fig. 2.2.
A E
D
C
B
Fig. 2.2. Optimal Binary Search Tree for the Example Instance
This problem sometimes appears in the literature in slightly modified ver-sions. One straightforward generalization introduces “dummy keys” represent-ing values not in X, which takes care of unsuccessful searches [10, p.357]. A second generalization considers arbitrary weights (e.g. real-valued weights) in-stead of probabilities. Another variation is to require that data items only be stored in the leaves of the tree, not in the internal nodes.
2.7 Optimal Covering Problem (COV)
This optimal covering problem is taken from [54, p.17]. Given are k different sized shrubs that need to be protected from frost. Assume the shrubs are sorted by size such that shrub 0 is the smallest, and shrub k− 1 is the largest.
The cost to manufacture a cover for shrub size i is denoted ci. However, due to manufacturing constraints, covers will be manufactured in no more than n different sizes, where n≤ k. Larger covers can protect smaller bushes. The objective is to select the n sizes which enable one to cover all shrubs at least cost.
Let j denote the number of cover sizes that have not been chosen yet and let l denote the largest shrub of the ones which are still under consideration.
The DP functional equation for this problem can be expressed as
f (j, l) =
min
d∈{j−2,...,l−1}{(l − d)cl+ f (j− 1, d)} if j > 1
(l + 1)cl if j = 1.
The goal is to compute f (n, k− 1).
Consider an instance of this problem with k = 10 shrubs, cover size costs (c0, . . . , c9) = (1, 4, 5, 7, 8, 12, 13, 18, 19, 21) and a manufacturing constraint of n = 3 cover sizes. Then the optimal policy is to order the manufacturing of cover sizes of 9, 6, and 4 (thus covering bushes 9, 8, and 7 with cover size 9, covering bushes 6 and 5 with cover size 6, and covering bushes 4 through 0 with cover size 4) at a total cost of f (3, 9) = 129.
2.8 Deadline Scheduling Problem (DEADLINE)
The deadline scheduling problem (see [22, pp.206–212] and [10, pp.399–401]) is that of choosing the optimal subset of a set of unit-time processes to be executed on a single processor, each process having a specified deadline and profit, where its profit is earned if the process completes execution before its deadline. The optimal subset is the one whose total earned profit is maximal.
The unit-time assumption means that each process completes execution one time unit after it starts. This scheduling problem can be solved using a greedy algorithm, but here we show that it can also be solved using DP in a fashion similar to many other scheduling problems. The state (k, S) is a stage number
k and a set S of processes that have not yet been considered. A decision d is a member of S. The next-state is (k + 1, S− {d}). The cost of choosing process d is either its profit or 0 depending upon whether its inclusion in the set of scheduled processes yields a “feasible” set, i.e. a set in which each process meets its deadline. This feasibility test is easier to implement if the set of processes is given in increasing order of deadlines. Assuming this ordering by deadlines, a chosen subset of processes would be executed in this sequential order; since we assumed unit-time processes, the j-th scheduled process ter-minates at time j. Thus, a sequence of processes S = {1, . . . , k} of size k is feasible if j ≤ tjfor 1≤ j ≤ k, in which case each process j in S can terminate before its deadline time tj.
The optimization problem can be solved using the DPFE f (k, S) = max
d∈S{c(d|S) + f(k + 1, S − {d})}, (2.11) where c(d|S) = wd if choosing to include d in the set of scheduled processes is “feasible”, else c(d|S) = 0. Our goal is to solve for f(1, S∗) given the base case f (k, S) = 0 for k = N + 1 or S =∅.
Assume a set of jobs S∗={0, 1, 2, 3, 4} having profits p = {10, 15, 20, 1, 5}
and deadline times t ={1, 2, 2, 3, 3}. Then f(1, {0, 1, 2, 3, 4}) = 15 + 20 + 5 + 0 + 0 = 40 for the optimal sequence of decisions d1= 1, d2= 2, d3 = 4, d4 = 0, d5= 3.
2.9 Discounted Profits Problem (DPP)
This Discounted Profits Problem (DPP) is described in [64, pp.779–780]. It is an intertemporal optimization problem that can be solved with DP. By incorporating the time value of money into the model, we get what is often referred to as a “discounted” DP problem.
Assume we are given a lake with an initial population of b1 fish at the beginning of year 1. The population at the beginning of year t is denoted bt. By selling xt fish during year t a revenue r(xt) is earned. The cost of catching these fish is c(xt, bt) and depends also on the number of fish in the lake. Fish reproduce, and this is modeled by a constant reproduction rate s — in [64, pp.779–780] it is assumed s = 1.2. That is, at the beginning of a year there are 20% more fish in the lake than at the end of the previous year. The finite planning horizon extends through the years 1, . . . , T during which we assume a constant interest rate y. The decision variable xtdenotes the number of fish to be caught and sold in year t. The goal is to maximize the net profit (in year 1 dollars) that can be earned during the years 1, . . . , T within the planning horizon. Typically for this type of decision problem there is a tradeoff of current benefits against future benefits.
A state in this DP model is a pair (t, b) representing the current year t and the fish population b at the beginning of the year. The DP functional equation becomes