Third Edition
4.5 The master method for solving recurrences
4.6.2 Floors and ceilings
To complete the proof of the master theorem, we must now extend our analysis to the situation in which floors and ceilings appear in the master recurrence, so that the recurrence is defined for all integers, not for just exact powers of b. Obtaining a lower bound on
T .n/ D aT .dn=be/ C f .n/ (4.25)
and an upper bound on
T .n/ D aT .bn=bc/ C f .n/ (4.26)
is routine, since we can push through the bounddn=be n=b in the first case to yield the desired result, and we can push through the boundbn=bc n=b in the second case. We use much the same technique to lower-bound the recurrence (4.26) as to upper-bound the recurrence (4.25), and so we shall present only this latter bound.
We modify the recursion tree of Figure 4.7 to produce the recursion tree in Fig-ure 4.8. As we go down in the recursion tree, we obtain a sequence of recursive invocations on the arguments
n ; dn=be ; ddn=be =be ; dddn=be =be =be ;
:::
Let us denote the j th element in the sequence by nj, where nj D
(n if j D 0 ;
dnj 1=be if j > 0 : (4.27)
…
Figure 4.8 The recursion tree generated by T .n/ D aT .dn=be/Cf .n/. The recursive argument nj
is given by equation (4.27).
Our first goal is to determine the depth k such that nk is a constant. Using the inequalitydxe x C 1, we obtain
n0 n ;
nj n
Letting j Dblogbnc, we obtain nblogbnc < n
and thus we see that at depthblogbnc, the problem size is at most a constant.
From Figure 4.8, we see that T .n/ D ‚.nlogba/ C
blogXbnc1 j D0
ajf .nj/ ; (4.28)
which is much the same as equation (4.21), except that n is an arbitrary integer and not restricted to be an exact power of b.
We can now evaluate the summation g.n/ D
blogXbnc1 j D0
ajf .nj/ (4.29)
from equation (4.28) in a manner analogous to the proof of Lemma 4.3. Beginning with case 3, if af .dn=be/ cf .n/ for n > bCb=.b1/, where c < 1 is a constant,
f .nj/ c which is similar to the corresponding proof of case 2, though the algebra is more intricate.
We have now proved the upper bounds in the master theorem for all integers n.
The proof of the lower bounds is similar.
Exercises
4.6-1 ?
Give a simple and exact expression for njin equation (4.27) for the case in which b is a positive integer instead of an arbitrary real number.
4.6-2 ?
Show that if f .n/ D ‚.nlogbalgkn/, where k 0, then the master recurrence has solution T .n/ D ‚.nlogbalgkC1n/. For simplicity, confine your analysis to exact powers of b.
4.6-3 ?
Show that case 3 of the master theorem is overstated, in the sense that the regularity condition af .n=b/ cf .n/ for some constant c < 1 implies that there exists a constant > 0 such that f .n/ D .nlogbaC/.
Problems
4-1 Recurrence examples
Give asymptotic upper and lower bounds for T .n/ in each of the following recur-rences. Assume that T .n/ is constant for n 2. Make your bounds as tight as possible, and justify your answers.
a. T .n/ D 2T .n=2/ C n4. b. T .n/ D T .7n=10/ C n.
c. T .n/ D 16T .n=4/ C n2. d. T .n/ D 7T .n=3/ C n2. e. T .n/ D 7T .n=2/ C n2. f. T .n/ D 2T .n=4/ Cp
n.
g. T .n/ D T .n 2/ C n2. 4-2 Parameter-passing costs
Throughout this book, we assume that parameter passing during procedure calls takes constant time, even if an N -element array is being passed. This assumption is valid in most systems because a pointer to the array is passed, not the array itself.
This problem examines the implications of three parameter-passing strategies:
1. An array is passed by pointer. Time D ‚.1/.
2. An array is passed by copying. Time D ‚.N /, where N is the size of the array.
3. An array is passed by copying only the subrange that might be accessed by the called procedure. Time D ‚.q p C 1/ if the subarray AŒp : : q is passed.
a. Consider the recursive binary search algorithm for finding a number in a sorted array (see Exercise 2.3-5). Give recurrences for the worst-case running times of binary search when arrays are passed using each of the three methods above, and give good upper bounds on the solutions of the recurrences. Let N be the size of the original problem and n be the size of a subproblem.
b. Redo part (a) for the MERGE-SORTalgorithm from Section 2.3.1.
4-3 More recurrence examples
Give asymptotic upper and lower bounds for T .n/ in each of the following recur-rences. Assume that T .n/ is constant for sufficiently small n. Make your bounds as tight as possible, and justify your answers.
a. T .n/ D 4T .n=3/ C n lg n.
b. T .n/ D 3T .n=3/ C n= lg n.
c. T .n/ D 4T .n=2/ C n2p n.
d. T .n/ D 3T .n=3 2/ C n=2.
e. T .n/ D 2T .n=2/ C n= lg n.
f. T .n/ D T .n=2/ C T .n=4/ C T .n=8/ C n.
g. T .n/ D T .n 1/ C 1=n.
h. T .n/ D T .n 1/ C lg n.
i. T .n/ D T .n 2/ C 1= lg n.
j. T .n/ Dp nT .p
n/ C n.
4-4 Fibonacci numbers
This problem develops properties of the Fibonacci numbers, which are defined by recurrence (3.22). We shall use the technique of generating functions to solve the Fibonacci recurrence. Define the generating function (or formal power se-ries)F as
F .´/ D X1 i D0
Fi´i
D 0 C ´ C ´2C 2´3C 3´4C 5´5C 8´6C 13´7C 21´8C ; where Fiis the i th Fibonacci number.
a. Show thatF .´/ D ´ C ´F .´/ C ´2F .´/.
b. Show that
5 for i > 0, rounded to the nearest integer.
(Hint: Observe thatˇˇyˇˇ< 1.) 4-5 Chip testing
Professor Diogenes has n supposedly identical integrated-circuit chips that in prin-ciple are capable of testing each other. The professor’s test jig accommodates two chips at a time. When the jig is loaded, each chip tests the other and reports whether it is good or bad. A good chip always reports accurately whether the other chip is good or bad, but the professor cannot trust the answer of a bad chip. Thus, the four possible outcomes of a test are as follows:
Chip A says Chip B says Conclusion
B is good A is good both are good, or both are bad B is good A is bad at least one is bad
B is bad A is good at least one is bad B is bad A is bad at least one is bad
a. Show that if more than n=2 chips are bad, the professor cannot necessarily de-termine which chips are good using any strategy based on this kind of pairwise test. Assume that the bad chips can conspire to fool the professor.
b. Consider the problem of finding a single good chip from among n chips, as-suming that more than n=2 of the chips are good. Show that bn=2c pairwise tests are sufficient to reduce the problem to one of nearly half the size.
c. Show that the good chips can be identified with ‚.n/ pairwise tests, assuming that more than n=2 of the chips are good. Give and solve the recurrence that describes the number of tests.
4-6 Monge arrays
An m n array A of real numbers is a Monge array if for all i , j , k, and l such that 1 i < k m and 1 j < l n, we have
AŒi; j C AŒk; l AŒi; l C AŒk; j :
In other words, whenever we pick two rows and two columns of a Monge array and consider the four elements at the intersections of the rows and the columns, the sum of the upper-left and lower-right elements is less than or equal to the sum of the lower-left and upper-right elements. For example, the following array is Monge:
10 17 13 28 23 17 22 16 29 23 24 28 22 34 24
11 13 6 17 7
45 44 32 37 23 36 33 19 21 6 75 66 51 53 34
a. Prove that an array is Monge if and only if for all i D 1; 2; :::; m 1 and j D 1; 2; :::; n 1, we have
AŒi; j C AŒi C 1; j C 1 AŒi; j C 1 C AŒi C 1; j :
(Hint: For the “if” part, use induction separately on rows and columns.) b. The following array is not Monge. Change one element in order to make it
Monge. (Hint: Use part (a).) 37 23 22 32
21 6 7 10
53 34 30 31
32 13 9 6
43 21 15 8
c. Let f .i / be the index of the column containing the leftmost minimum element of row i . Prove that f .1/ f .2/ f .m/ for any m n Monge array.
d. Here is a description of a divide-and-conquer algorithm that computes the left-most minimum element in each row of an m n Monge array A:
Construct a submatrix A0of A consisting of the even-numbered rows of A.
Recursively determine the leftmost minimum for each row of A0. Then compute the leftmost minimum in the odd-numbered rows of A.
Explain how to compute the leftmost minimum in the odd-numbered rows of A (given that the leftmost minimum of the even-numbered rows is known) in O.m C n/ time.
e. Write the recurrence describing the running time of the algorithm described in part (d). Show that its solution is O.m C n log m/.
Chapter notes
Divide-and-conquer as a technique for designing algorithms dates back to at least 1962 in an article by Karatsuba and Ofman [194]. It might have been used well be-fore then, however; according to Heideman, Johnson, and Burrus [163], C. F. Gauss devised the first fast Fourier transform algorithm in 1805, and Gauss’s formulation breaks the problem into smaller subproblems whose solutions are combined.
The maximum-subarray problem in Section 4.1 is a minor variation on a problem studied by Bentley [43, Chapter 7].
Strassen’s algorithm [325] caused much excitement when it was published in 1969. Before then, few imagined the possibility of an algorithm asymptotically faster than the basic SQUARE-MATRIX-MULTIPLY procedure. The asymptotic upper bound for matrix multiplication has been improved since then. The most asymptotically efficient algorithm for multiplying n n matrices to date, due to Coppersmith and Winograd [78], has a running time of O.n2:376/. The best lower bound known is just the obvious .n2/ bound (obvious because we must fill in n2 elements of the product matrix).
From a practical point of view, Strassen’s algorithm is often not the method of choice for matrix multiplication, for four reasons:
1. The constant factor hidden in the ‚.nlg 7/ running time of Strassen’s algo-rithm is larger than the constant factor in the ‚.n3/-time SQUARE-MATRIX -MULTIPLY procedure.
2. When the matrices are sparse, methods tailored for sparse matrices are faster.
3. Strassen’s algorithm is not quite as numerically stable as SQUARE-MATRIX -MULTIPLY. In other words, because of the limited precision of computer arith-metic on noninteger values, larger errors accumulate in Strassen’s algorithm than in SQUARE-MATRIX-MULTIPLY.
4. The submatrices formed at the levels of recursion consume space.
The latter two reasons were mitigated around 1990. Higham [167] demonstrated that the difference in numerical stability had been overemphasized; although Strassen’s algorithm is too numerically unstable for some applications, it is within acceptable limits for others. Bailey, Lee, and Simon [32] discuss techniques for reducing the memory requirements for Strassen’s algorithm.
In practice, fast matrix-multiplication implementations for dense matrices use Strassen’s algorithm for matrix sizes above a “crossover point,” and they switch to a simpler method once the subproblem size reduces to below the crossover point. The exact value of the crossover point is highly system dependent. Analyses that count operations but ignore effects from caches and pipelining have produced crossover points as low as n D 8 (by Higham [167]) or n D 12 (by Huss-Lederman et al. [186]). D’Alberto and Nicolau [81] developed an adaptive scheme, which determines the crossover point by benchmarking when their software package is installed. They found crossover points on various systems ranging from n D 400 to n D 2150, and they could not find a crossover point on a couple of systems.
Recurrences were studied as early as 1202 by L. Fibonacci, for whom the Fi-bonacci numbers are named. A. De Moivre introduced the method of generating functions (see Problem 4-4) for solving recurrences. The master method is adapted from Bentley, Haken, and Saxe [44], which provides the extended method justified by Exercise 4.6-2. Knuth [209] and Liu [237] show how to solve linear recurrences using the method of generating functions. Purdom and Brown [287] and Graham, Knuth, and Patashnik [152] contain extended discussions of recurrence solving.
Several researchers, including Akra and Bazzi [13], Roura [299], Verma [346], and Yap [360], have given methods for solving more general divide-and-conquer recurrences than are solved by the master method. We describe the result of Akra and Bazzi here, as modified by Leighton [228]. The Akra-Bazzi method works for recurrences of the form
T .x/ D (
‚.1/ if 1 x x0;
Pk
i D1aiT .bix/ C f .x/ if x > x0; (4.30) where
x 1 is a real number,
x0is a constant such that x0 1=biand x0 1=.1 bi/ for i D 1; 2; : : : ; k,
ai is a positive constant for i D 1; 2; : : : ; k,
biis a constant in the range 0 < bi < 1 for i D 1; 2; : : : ; k,
k 1 is an integer constant, and
f .x/ is a nonnegative function that satisfies the polynomial-growth condi-tion: there exist positive constants c1 and c2 such that for all x 1, for i D 1; 2; : : : ; k, and for all u such that bix u x, we have c1f .x/ f .u/ c2f .x/. (If jf0.x/j is upper-bounded by some polynomial in x, then f .x/ satisfies the polynomial-growth condition. For example, f .x/ D x˛lgˇx satisfies this condition for any real constants ˛ and ˇ.)
Although the master method does not apply to a recurrence such as T .n/ D T .bn=3c/ C T .b2n=3c/ C O.n/, the Akra-Bazzi method does. To solve the re-currence (4.30), we first find the unique real number p such thatPk
i D1aibip D 1.
(Such a p always exists.) The solution to the recurrence is then T .n/ D ‚
xp
1 C
Z x 1
f .u/
upC1 du
:
The Akra-Bazzi method can be somewhat difficult to use, but it serves in solving recurrences that model division of the problem into substantially unequally sized subproblems. The master method is simpler to use, but it applies only when sub-problem sizes are equal.
Algorithms
This chapter introduces probabilistic analysis and randomized algorithms. If you are unfamiliar with the basics of probability theory, you should read Appendix C, which reviews this material. We shall revisit probabilistic analysis and randomized algorithms several times throughout this book.