Third Edition
4.3 The substitution method for solving recurrences
Now that we have seen how recurrences characterize the running times of divide-and-conquer algorithms, we will learn how to solve recurrences. We start in this section with the “substitution” method.
The substitution method for solving recurrences comprises two steps:
1. Guess the form of the solution.
2. Use mathematical induction to find the constants and show that the solution works.
We substitute the guessed solution for the function when applying the inductive hypothesis to smaller values; hence the name “substitution method.” This method is powerful, but we must be able to guess the form of the answer in order to apply it.
We can use the substitution method to establish either upper or lower bounds on a recurrence. As an example, let us determine an upper bound on the recurrence
T .n/ D 2T .bn=2c/ C n ; (4.19)
which is similar to recurrences (4.3) and (4.4). We guess that the solution is T .n/ D O.n lg n/. The substitution method requires us to prove that T .n/ cn lg n for an appropriate choice of the constant c > 0. We start by assuming that this bound holds for all positive m < n, in particular for m Dbn=2c, yielding T .bn=2c/ c bn=2c lg.bn=2c/. Substituting into the recurrence yields
T .n/ 2.c bn=2c lg.bn=2c// C n
cn lg.n=2/ C n D cn lg n cn lg 2 C n D cn lg n cn C n
cn lg n ;
where the last step holds as long as c 1.
Mathematical induction now requires us to show that our solution holds for the boundary conditions. Typically, we do so by showing that the boundary condi-tions are suitable as base cases for the inductive proof. For the recurrence (4.19), we must show that we can choose the constant c large enough so that the bound T .n/ cn lg n works for the boundary conditions as well. This requirement can sometimes lead to problems. Let us assume, for the sake of argument, that T .1/ D 1 is the sole boundary condition of the recurrence. Then for n D 1, the bound T .n/ cn lg n yields T .1/ c1 lg 1 D 0, which is at odds with T .1/ D 1.
Consequently, the base case of our inductive proof fails to hold.
We can overcome this obstacle in proving an inductive hypothesis for a spe-cific boundary condition with only a little more effort. In the recurrence (4.19), for example, we take advantage of asymptotic notation requiring us only to prove T .n/ cn lg n for n n0, where n0 is a constant that we get to choose. We keep the troublesome boundary condition T .1/ D 1, but remove it from consid-eration in the inductive proof. We do so by first observing that for n > 3, the recurrence does not depend directly on T .1/. Thus, we can replace T .1/ by T .2/
and T .3/ as the base cases in the inductive proof, letting n0 D 2. Note that we make a distinction between the base case of the recurrence (n D 1) and the base cases of the inductive proof (n D 2 and n D 3). With T .1/ D 1, we derive from the recurrence that T .2/ D 4 and T .3/ D 5. Now we can complete the inductive proof that T .n/ cn lg n for some constant c 1 by choosing c large enough so that T .2/ c2 lg 2 and T .3/ c3 lg 3. As it turns out, any choice of c 2 suffices for the base cases of n D 2 and n D 3 to hold. For most of the recurrences we shall examine, it is straightforward to extend boundary conditions to make the inductive assumption work for small n, and we shall not always explicitly work out the details.
Making a good guess
Unfortunately, there is no general way to guess the correct solutions to recurrences.
Guessing a solution takes experience and, occasionally, creativity. Fortunately, though, you can use some heuristics to help you become a good guesser. You can also use recursion trees, which we shall see in Section 4.4, to generate good guesses.
If a recurrence is similar to one you have seen before, then guessing a similar solution is reasonable. As an example, consider the recurrence
T .n/ D 2T .bn=2c C 17/ C n ;
which looks difficult because of the added “17” in the argument to T on the right-hand side. Intuitively, however, this additional term cannot substantially affect the
solution to the recurrence. When n is large, the difference between bn=2c and bn=2c C 17 is not that large: both cut n nearly evenly in half. Consequently, we make the guess that T .n/ D O.n lg n/, which you can verify as correct by using the substitution method (see Exercise 4.3-6).
Another way to make a good guess is to prove loose upper and lower bounds on the recurrence and then reduce the range of uncertainty. For example, we might start with a lower bound of T .n/ D .n/ for the recurrence (4.19), since we have the term n in the recurrence, and we can prove an initial upper bound of T .n/ D O.n2/. Then, we can gradually lower the upper bound and raise the lower bound until we converge on the correct, asymptotically tight solution of T .n/ D ‚.n lg n/.
Subtleties
Sometimes you might correctly guess an asymptotic bound on the solution of a recurrence, but somehow the math fails to work out in the induction. The problem frequently turns out to be that the inductive assumption is not strong enough to prove the detailed bound. If you revise the guess by subtracting a lower-order term when you hit such a snag, the math often goes through.
Consider the recurrence
T .n/ D T .bn=2c/ C T .dn=2e/ C 1 :
We guess that the solution is T .n/ D O.n/, and we try to show that T .n/ cn for an appropriate choice of the constant c. Substituting our guess in the recurrence, we obtain
T .n/ c bn=2c C c dn=2e C 1 D cn C 1 ;
which does not imply T .n/ cn for any choice of c. We might be tempted to try a larger guess, say T .n/ D O.n2/. Although we can make this larger guess work, our original guess of T .n/ D O.n/ is correct. In order to show that it is correct, however, we must make a stronger inductive hypothesis.
Intuitively, our guess is nearly right: we are off only by the constant 1, a lower-order term. Nevertheless, mathematical induction does not work unless we prove the exact form of the inductive hypothesis. We overcome our difficulty by subtracting a lower-order term from our previous guess. Our new guess is T .n/ cn d , where d 0 is a constant. We now have
T .n/ .c bn=2c d / C .c dn=2e d / C 1 D cn 2d C 1
cn d ;
as long as d 1. As before, we must choose the constant c large enough to handle the boundary conditions.
You might find the idea of subtracting a lower-order term counterintuitive. Af-ter all, if the math does not work out, we should increase our guess, right?
Not necessarily! When proving an upper bound by induction, it may actually be more difficult to prove that a weaker upper bound holds, because in order to prove the weaker bound, we must use the same weaker bound inductively in the proof.
In our current example, when the recurrence has more than one recursive term, we get to subtract out the lower-order term of the proposed bound once per recursive term. In the above example, we subtracted out the constant d twice, once for the T .bn=2c/ term and once for the T .dn=2e/ term. We ended up with the inequality T .n/ cn 2d C 1, and it was easy to find values of d to make cn 2d C 1 be less than or equal to cn d .
Avoiding pitfalls
It is easy to err in the use of asymptotic notation. For example, in the recur-rence (4.19) we can falsely “prove” T .n/ D O.n/ by guessing T .n/ cn and then arguing
T .n/ 2.c bn=2c/ C n
cn C n
D O.n/ ; wrong!!
since c is a constant. The error is that we have not proved the exact form of the inductive hypothesis, that is, that T .n/ cn. We therefore will explicitly prove that T .n/ cn when we want to show that T .n/ D O.n/.
Changing variables
Sometimes, a little algebraic manipulation can make an unknown recurrence simi-lar to one you have seen before. As an example, consider the recurrence
T .n/ D 2T p n˘
C lg n ;
which looks difficult. We can simplify this recurrence, though, with a change of variables. For convenience, we shall not worry about rounding off values, such asp
n, to be integers. Renaming m D lg n yields T .2m/ D 2T .2m=2/ C m :
We can now rename S.m/ D T .2m/ to produce the new recurrence S.m/ D 2S.m=2/ C m ;
which is very much like recurrence (4.19). Indeed, this new recurrence has the same solution: S.m/ D O.m lg m/. Changing back from S.m/ to T .n/, we obtain T .n/ D T .2m/ D S.m/ D O.m lg m/ D O.lg n lg lg n/ :
Exercises
4.3-1
Show that the solution of T .n/ D T .n 1/ C n is O.n2/.
4.3-2
Show that the solution of T .n/ D T .dn=2e/ C 1 is O.lg n/.
4.3-3
We saw that the solution of T .n/ D 2T .bn=2c/ C n is O.n lg n/. Show that the so-lution of this recurrence is also .n lg n/. Conclude that the soso-lution is ‚.n lg n/.
4.3-4
Show that by making a different inductive hypothesis, we can overcome the diffi-culty with the boundary condition T .1/ D 1 for recurrence (4.19) without adjusting the boundary conditions for the inductive proof.
4.3-5
Show that ‚.n lg n/ is the solution to the “exact” recurrence (4.3) for merge sort.
4.3-6
Show that the solution to T .n/ D 2T .bn=2c C 17/ C n is O.n lg n/.
4.3-7
Using the master method in Section 4.5, you can show that the solution to the recurrence T .n/ D 4T .n=3/ C n is T .n/ D ‚.nlog34/. Show that a substitution proof with the assumption T .n/ cnlog34 fails. Then show how to subtract off a lower-order term to make a substitution proof work.
4.3-8
Using the master method in Section 4.5, you can show that the solution to the recurrence T .n/ D 4T .n=2/ C n2 is T .n/ D ‚.n2/. Show that a substitution proof with the assumption T .n/ cn2 fails. Then show how to subtract off a lower-order term to make a substitution proof work.
4.3-9
Solve the recurrence T .n/ D 3T .p
n/ C log n by making a change of variables.
Your solution should be asymptotically tight. Do not worry about whether values are integral.