Complexity of Algorithms

SECTION 3.3 Complexity of Algorithms

Some of these exercises involve analysis of algorithms, as was done in the examples in this section. These are a matter of carefully counting the operations of interest, usually in the worst case. Some of the others are algebra exercises that display the results of the analysis in real terms-the number of years of computer time, for example, required to solve a large problem. Horner's method for evaluating a polynomial, given in Exercise 14, is a nice trick to know. It is extremely handy for polynomial evaluation on a pocket calculator

(especially if the calculator is so cheap that it does not use the usual precedence rules).

1. The statement

t

:=

t +

ij is executed just 12 times, so the number of operations is 0(1). (Specifically, there are just 24 additions or multiplications.)

3. The nesting of the loops implies that the assignment statement is executed roughly n² /2 times. Therefore the number of operations is 0( n²) .

5. Assuming that the algorithm given to find the smallest element of a list is identical to Algorithm 1 in Section 3.1, except that the inequality is reversed (and the name max replaced by the name min), the analysis will be identical to the analysis given in Example 1 in the current section. In particular, there will be 2n - 1 comparisons needed, counting the bookkeeping for the loop.

7. The linear search would find this element after at most 9 comparisons ( 4 to determine that we have not yet finished with the while loop, 4 more to determine if we have located the desired element yet, and 1 to set the value of location). Binary search, according to Example 3, will take 2 log 32

+

2

=

2 · 5

+

2

=

12 comparisons.

Since 9

<

12, the linear search will be faster, in terms of comparisons.

9. The algorithm simply scans the bits one at a time. Thus clearly O(n) comparisons are required (perhaps one for bookkeeping and one for looking at the ith bit, for each i from 1 to n ).

11. a) We can express the suggested algorithm in pseudocode as follows. Notice that the Boolean variable disjoint is set to true at the beginning of the comparison of sets S, and S_{1 ,} and becomes false if and when we find an element common to those two sets. If disjoint is never set to false, then we have found a disjoint pair, and answer is set to true. This process is repeated for each pair of sets (controlled by the outer two loops).

procedure disjointpair(S1 , S2 , . . . , Sn : subsets of {1, 2, ... , n}) answer:= false

for i := 1 ton

for j := i

+

1 ton disjoint := true fork:= 1 ton

return answer

if k E Si and k E S₁ then disjoint:= false if disjoint then answer := true

b) The three nested loops imply that the elementhood test needs to be applied O(n³) times.

13. a) Here we have n = 2, a0 = 1, a1 = 1, a2 = 3, and c = 2. Initially, we set power equal to 1 and y equal to 1. The first time through the for loop (with i = 1 ) , power becomes 2 and so y becomes 1

+

1 · 2 = 3.

The second and final time through the loop, power becomes 2 · 2

=

4 and y becomes 3

+

3 · 4

=

15. Thus the value of the polynomial at x

=

2 is 15.

b) Each pass through the loop requires two multiplications and one addition. Therefore there are a total of 2n multiplications and n additions in all.

104 Chapter 3 Algorithms

15. This is an exercise in algebra, numerical analysis (for some of the parts), and using a calculator. Since each bit operation requires 10-⁹ seconds, we want to know for what value of n there will be at most 10⁹ bit operations required. Thus we need to set the expression equal to 10⁹, solve for n, and round down if necessary.

a) Solving logn = 10⁹, we get (recalling that "log'' means logarithm base 2) n = 2¹⁰⁹. By taking log₁₀ of both sides, we find that this number is approximately equal to 10³oo,ooo,ooo. Obviously we do not want to write out the answer explicitly!

b) Clearly n

=

10⁹.

c) Solving n log n

=

10⁹ is not trivial. There is no good formula for solving such transcendental equations.

An algorithm that works well with a calculator is to rewrite the equation as n

=

10⁹ /log n, enter a random starting value, say n = 2, and repeatedly calculate a new value of n. Thus we would obtain, in succession, n

=

10⁹/log2

=

10⁹, n

=

10⁹/log(10⁹):::::; 33,447,777.3, n

=

10⁹/log(33,447,777.3):::::; 40,007,350.14, n

=

10⁹ /log( 40,007,350.14) :::::; 39,598,061.08, and so on. After a few more iterations, the numbers stabilize at approximately 39,620,077. 73, so the answer is 39,620,077.

d) Solving n²

=

10⁹ gives n

=

10⁴·5, which is 31,622 when rounded down.

e) Solving 2n

=

10⁹ gives n

=

log(10⁹):::::; 29.9. Rounding down gives the answer, 29.

f)

The quickest way to find the largest value of n such that n! :::; 10⁹ is simply to try a few values of n. We find that 12! :::::; 4.8 x 10⁸ while 13! :::::; 6.2 x 10⁹, so the answer is 12.

17. If each bit operation takes 10-¹² second, then we can carry out 10¹² bit operations per second, and therefore 60 · 10¹² bit operations per minute. Therefore in each case we want to solve the equation f(n)

=

60 · 10¹² for n and round down to an integer. Obviously a calculator will come in handy here.

60 10¹²

a) If loglogn

=

60 · 10¹², then n

=

2² , which is an unfathomably huge number.

b) If log n = 60 · 10¹², then n = 2^{60 1012} , which is still an unfathomably huge number.

c) If (log n )²

=

60 · 10¹², then log n

= V6Q ·

10⁶, so n

=

L 2v'60·l0⁵

J ,

which is still an extremely large number (it has over 2 million digits).

d) If l,000,000n

=

60 · 10¹², then n

=

60 · 10⁶

=

60,000,000.

e) If n²

=

60 · 10¹², then n

= l V66 ·

10⁶

J =

7,745,966.

f) If 2n

=

60 · 10¹², then n

=

Llog(60 · 10¹²

)J =

Llog 60

+

12log10

J =

45. (Remember, we are taking log to the base 2.)

g) If 2n²

=

60 · 10¹², then n

=

h/log(60 · 10¹²

)J = l

yflog 60

+

12log10

J =

6. (Remember, we are taking log to the base 2.)

19. In each case, we just multiply the number of seconds per operation by the number of operations (namely 2^{50 ).}

To convert seconds to minutes, we divide by 60; to convert minutes to hours, we divide by 60 again. To convert hours to days, we divide by 24; to convert days to years, we divide by 365 ~ .

a) 2⁵⁰ x 10-⁶

=

1,125,899,907 seconds:::::; 36 years b) 2⁵⁰ x 10-⁹

=

1,125,899.907 seconds:::::; 13 days c) 2⁵⁰ x 10-¹² = 1,125.899907 seconds:::::; 19 minutes

21. In each case we want to compare the function evaluated at n

+

1 to the function evaluated at n. The most desirable form of the comparison (subtraction or division) will vary.

a) Notice that log( n

+

1) - log n

=

log "~¹ . If n is at all large, the fraction in this expression is approximately equal to 1, and therefore the expression is approximately equal to 0. In other words, hardly any extra time is required. For example, in going from n = 10 to n = 11, the number of extra milliseconds is log 11/10:::::; 0.14.

b) lOO(n

+

1) - lOOn = 100. One hundred extra milliseconds are required, independent of n.

c) Because (n

+

1)² - n²

=

2n

+

1, we conclude that 2n

+

1 additional milliseconds are needed for the larger problem.

Section 3.3 Complexity of Algorithms 105 d) Because (n + 1)³ - n³

=

3n² + 3n + 1, we conclude that 3n² + 3n + 1 additional milliseconds are needed for the larger problem.

e) This time it makes more sense to use a ratio comparison, rather than a difference comparison. Because 2n+i /2n

=

2, we see that twice as much time is required for the larger problem.

f)

Because 2<n+¹⁾² ;2n² = 2²n+^1, we see that 2²¹¹+1 times as many milliseconds are required for the larger problem.

g) Because (n + 1)!/n!

=

n + 1, we see that n + 1 times as many milliseconds are required for the larger problem. For example, a problem with n

=

10 takes ten times as long as a problem with n = 9.

23. If the element is not in the list, then 2n + 2 comparisons are needed: two for each pass through the loop, one more to get out of the loop, and one more for the statement just after the loop. If the element is in the list, say as the ith element, then we need to enter the loop i times, each time costing two comparisons, and use one comparison for the final assignment of location. Thus 2i + 1 comparisons are needed. We need to average the numbers 2i + 1 (for i from 1 to n), to find the average number of comparisons needed for a successful search. This is (3 + 5 + · · · + (2n + 1))/n

=

(n + (2 + 4 + · · · + 2n))/n

=

(n + 2(1+2 + · · · + n))/n

=

(n+2(n(n+l)/2))/n

=

n+2. Finally, we average the 2n+2 comparisons for the unsuccessful search with this average n + 2 comparisons for a successful search to obtain a grand average of (2n + 2 + n + 2)/2 = (3n + 4)/2 comparisons.

25. We will count comparisons of elements in the list to x. (This ignores comparisons of subscripts, but since we are only interested in a big-0 analysis, no harm is done.) Furthermore, we will assume that the number of elements in the list is a power of 3, say n

=

3k . Just as in the case of binary search, we need to determine the maximum number of times the while loop is iterated. Each pass through the loop cuts the number of elements still being considered (those whose subscripts are from i to j) by a factor of 3. Therefore after k iterations, the active portion of the list will have length 1; that is, we will have i

=

j. The loop terminates at this point. Now each iteration of the loop requires two comparisons in the worst case (one with au and one with az). Two more comparisons are needed at the end. Therefore the number of comparisons is 2k + 2, which is O(k). But k

=

^log3n, which is O(logn) since logarithms to different bases differ only by multiplicative constants, so the time complexity of this algorithm (in all cases, not just the worst case) is O(logn).

27. The algorithm we gave for finding a mode essentially just goes through the list once, doing a little bookkeeping at each step. In particular, between any two successive executions of the statement i := i + 1 there are at most about six operations (such as comparing count with modecount, or reinitializing value). Therefore at most about 6n steps are done in all, so the time complexity in all cases is O(n).

29. The worst case is that in which we do not find any term equal to some previous term. In that case, we need to go through all the terms a2 through an, and for each of those, we need to go through all the terms occurring prior to that term. Thus the inner loop of our algorithm is executed once for i

=

2 (namely for j = 1 ), twice for i

=

3 (namely for j

=

1 and j = 2), three times for i

=

4, and so on, up to n - 1 times for i = n.

Thus the number of comparisons that need to be made in the inner loop is 1 + 2 + 3 + · · · + ( n - 1) . As was mentioned in this section (and will be shown in Section 5.1), that sum is ( n - 1) ( n - 1 + 1) /2, which is clearly O(n²) but no better. Bookkeeping details do not increase this estimate.

31. We needed to go through the sequence only once, making one comparison of terms (and two bookkeeping comparisons) until we found the desired term (or had exhausted the list). Thus this algorithm's time complexity is clearly 0( n).

33. We had to read the string simultaneously from the front and the back and compare characters to make sure they were equal. Thus we will need at least Ln/2J comparisons in the worst case, which is O(n).

106 Chapter 3 Algorithms

35. Since we are doing binary search to find the correct location of the jth element among the first j - 1 elements, which are already sorted, we need O(logn) comparisons for each element. Therefore we use O(nlogn) comparisons in all. The cost of swapping of items to make room for the insertions is O(n²), however (the originally first item may need to move n - 1 places, the second item n - 2 places, and so on).

37. There are 2n subsets of the n talks. For each one, we need to compare the times to see whether they overlap.

To compare a pair of talks for overlap takes 0(1) time (there is no overlap if and only if the first talk's start time exceeds the second talk's end time, or vice versa, so just two comparisons are needed). There are O(n2 )

pairs of talks to compare in a typical subset. (We also need to compare the size of this subset with the size of the best subset found so far, but the time for that is negligible.) Therefore the total complexity is O(n²2n).

39. a) The linear search algorithm uses about n comparisons for a list of length n, and 2n comparisons for a list of length 2n. Therefore the number of comparisons, like the size of the list, doubles.

b) The binary search algorithm uses about log n comparisons for a list of length n, and log(2n) = log 2 + log n

=

1 + log n comparisons for a list of length 2n. Therefore the number of comparisons increases by about 1.

41. We just use Algorithm 1, where A and B are now n x n upper triangular matrices, by replacing m by n in line 1, and having q iterate only from i to j, rather than from 1 to k.

43. See the solution to Exercise 42. Looking at the nested loops, we see that the number of multiplications is given by the following expression: [1+2 + · · · + n] + [1+2 + · · · (n - 1)] + · · · + [l]. To simplify this, we need the fact (from Table 2 in Section 2.4) that the sum of the first k positive integers is k(k + 1)/2, as well as the result from Exercise 15 in Section 5.1, which says that the sum 1·2+2 · 3 + 3 · 4 + · · · + k(k + 1) equals k(k + l)(k + 1)/3. Doing the algebra, we obtain n(n + l)(n + 2)/6 as the total number of multiplications.

45. There are five different ways to perform this multiplication:

(AB)(CD), ((AB)C)D, A(B(CD)), (A(BC))D, A((BC)D).

We can use the result stated in the preamble to find the numbers of multiplications needed in these five cases.

For example, in the first case we need 30 · 10 · 40 = 12,000 multiplications to compute the 30 x 40 matrix AB, 40 · 50 · 30 = 60,000 multiplications to compute the 40 x 30 matrix CD, and then 30 · 40 · 30 = 36,000 multiplications to multiply these two matrices together to obtain the final answer. This gives a total of 12,000 + 60,000 + 36,000

=

108,000 multiplications. Similar calculations for the other four cases yield 30 · 10 · 40 + 30 · 40 · 50 + 30 · 50 · 30 = 117,00, 40 · 50 · 30 + 10 · 40 · 30 + 30 · 10 · 30 = 81,000, 10 · 40 · 50 + 30 · 10 · 50 + 30 · 50 · 30 = 80,000, and 10 · 40 · 50 + 10 · 50 · 30 + 30 · 10 · 30 = 44,000, respectively. The winner is therefore A((BC)D), requiring 44,000 multiplications. Note that the worst arrangement requires 117,00 multiplications; it will take more than twice as long.

Review Questions 107

在文檔中 Student's Solutions Guide (頁 112-116)