Standard notations and common functions

This section reviews some standard mathematical functions and notations and ex-plores the relationships among them. It also illustrates the use of the asymptotic notations.

Monotonicity

A function f .n/ is monotonically increasing if m  n implies f .m/  f .n/.

Similarly, it is monotonically decreasing if m  n implies f .m/  f .n/. A function f .n/ is strictly increasing if m < n implies f .m/ < f .n/ and strictly decreasing if m < n implies f .m/ > f .n/.

Floors and ceilings

For any real number x, we denote the greatest integer less than or equal to x bybxc (read “the floor of x”) and the least integer greater than or equal to x bydxe (read

“the ceiling of x”). For all real x,

x  1 < bxc  x  dxe < x C 1 : (3.3)

The floor function f .x/ Dbxc is monotonically increasing, as is the ceiling func-tion f .x/ Ddxe.

Modular arithmetic

For any integer a and any positive integer n, the value a mod n is the remainder (or residue) of the quotient a=n:

a mod n D a  n ba=nc : (3.8)

It follows that

0  a mod n < n : (3.9)

Given a well-defined notion of the remainder of one integer when divided by an-other, it is convenient to provide special notation to indicate equality of remainders.

If .a mod n/ D .b mod n/, we write a  b .mod n/ and say that a is equivalent to b, modulo n. In other words, a  b .mod n/ if a and b have the same remain-der when divided by n. Equivalently, a  b .mod n/ if and only if n is a divisor of b  a. We write a 6 b .mod n/ if a is not equivalent to b, modulo n.

Polynomials

Given a nonnegative integer d , a polynomial inn of degree d is a function p.n/

of the form p.n/ D

Xd i D0

ainⁱ ;

where the constants a0; a1; : : : ; ad are the coefficients of the polynomial and ad ¤ 0. A polynomial is asymptotically positive if and only if ad > 0. For an asymptotically positive polynomial p.n/ of degree d , we have p.n/ D ‚.n^d/. For any real constant a  0, the function n^a is monotonically increasing, and for any real constant a  0, the function n^a is monotonically decreasing. We say that a function f .n/ is polynomially bounded if f .n/ D O.n^k/ for some constant k.

Exponentials

For all real a > 0, m, and n, we have the following identities:

a⁰ D 1 ; a¹ D a ; a^1 D 1=a ; .a^m/ⁿ D a^mn; .a^m/ⁿ D .aⁿ/^m;

a^maⁿ D a^mCn:

For all n and a  1, the function aⁿ is monotonically increasing in n. When convenient, we shall assume 0⁰D 1.

We can relate the rates of growth of polynomials and exponentials by the fol-lowing fact. For all real constants a and b such that a > 1,

n!1lim n^b

aⁿ D 0 ; (3.10)

from which we can conclude that n^b D o.aⁿ/ :

Thus, any exponential function with a base strictly greater than 1 grows faster than any polynomial function.

Using e to denote 2:71828 : : :, the base of the natural logarithm function, we have for all real x,

e^x D 1 C x C x² 2Š Cx³

3Š C    D X1 i D0

xⁱ

i Š ; (3.11)

where “Š” denotes the factorial function defined later in this section. For all real x, we have the inequality

e^x  1 C x ; (3.12)

where equality holds only when x D 0. Whenjxj  1, we have the approximation

1 C x  e^x  1 C x C x²: (3.13)

When x ! 0, the approximation of e^x by 1 C x is quite good:

e^x D 1 C x C ‚.x²/ :

(In this equation, the asymptotic notation is used to describe the limiting behavior as x ! 0 rather than as x ! 1.) We have for all x,

n!1lim

 1 C x

n

n

D e^x: (3.14)

Logarithms

We shall use the following notations:

lg n D log₂n (binary logarithm) , ln n D log_en (natural logarithm) , lg^kn D .lg n/^k (exponentiation) , lg lg n D lg.lg n/ (composition) .

An important notational convention we shall adopt is that logarithm functions will apply only to the next term in the formula, so that lg n C k will mean .lg n/ C k and not lg.n C k/. If we hold b > 1 constant, then for n > 0, the function log_bn is strictly increasing.

For all real a > 0, b > 0, c > 0, and n, a D b^logba;

log_c.ab/ D log_ca C log_cb ; log_baⁿ D n log_ba ;

log_ba D log_ca

log_cb ; (3.15)

log_b.1=a/ D  log_ba ; log_ba D 1

log_ab ;

a^logbc D c^logba; (3.16)

where, in each equation above, logarithm bases are not 1.

By equation (3.15), changing the base of a logarithm from one constant to an-other changes the value of the logarithm by only a constant factor, and so we shall often use the notation “lg n” when we don’t care about constant factors, such as in O-notation. Computer scientists find 2 to be the most natural base for logarithms because so many algorithms and data structures involve splitting a problem into two parts.

There is a simple series expansion for ln.1 C x/ whenjxj < 1:

ln.1 C x/ D x x² 2 Cx³

3 x⁴ 4 Cx⁵

5     : We also have the following inequalities for x > 1:

x

1 C x  ln.1 C x/  x ; (3.17)

where equality holds only for x D 0.

We say that a function f .n/ is polylogarithmically bounded if f .n/ D O.lg^kn/

for some constant k. We can relate the growth of polynomials and polylogarithms by substituting lg n for n and 2^afor a in equation (3.10), yielding

n!1lim From this limit, we can conclude that lg^bn D o.n^a/

for any constant a > 0. Thus, any positive polynomial function grows faster than any polylogarithmic function.

Factorials

The notation nŠ (read “n factorial”) is defined for integers n  0 as nŠ D

(

1 if n D 0 ;

n  .n  1/Š if n > 0 : Thus, nŠ D 1  2  3    n.

A weak upper bound on the factorial function is nŠ  nⁿ, since each of the n terms in the factorial product is at most n. Stirling’s approximation,

nŠ Dp

where e is the base of the natural logarithm, gives us a tighter upper bound, and a lower bound as well. As Exercise 3.2-3 asks you to prove,

nŠ D o.nⁿ/ ; nŠ D !.2ⁿ/ ;

lg.nŠ/ D ‚.n lg n/ ; (3.19)

where Stirling’s approximation is helpful in proving equation (3.19). The following equation also holds for all n  1:

nŠ Dp

2 nn e

n

e^˛n (3.20)

where 1

12n C 1 < ˛_n< 1

12n : (3.21)

Functional iteration

We use the notation f^{.i /}.n/ to denote the function f .n/ iteratively applied i times to an initial value of n. Formally, let f .n/ be a function over the reals. For non-negative integers i , we recursively define

f^{.i /}.n/ D

(n if i D 0 ;

f .f^{.i 1/}.n// if i > 0 :

For example, if f .n/ D 2n, then f^{.i /}.n/ D 2ⁱn.

The iterated logarithm function

We use the notation lg^n (read “log star of n”) to denote the iterated logarithm, de-fined as follows. Let lg^{.i /}n be as defined above, with f .n/ D lg n. Because the log-arithm of a nonpositive number is undefined, lg^{.i /}n is defined only if lg^{.i 1/}n > 0.

Be sure to distinguish lg^{.i /}n (the logarithm function applied i times in succession, starting with argument n) from lgⁱn (the logarithm of n raised to the i th power).

Then we define the iterated logarithm function as lg^n D min˚

i  0 W lg^{.i /}n  1 :

The iterated logarithm is a very slowly growing function:

lg^2 D 1 ; lg^4 D 2 ; lg^16 D 3 ; lg^65536 D 4 ; lg^.2⁶⁵⁵³⁶/ D 5 :

Since the number of atoms in the observable universe is estimated to be about 10⁸⁰, which is much less than 2⁶⁵⁵³⁶, we rarely encounter an input size n such that lg^n > 5.

Fibonacci numbers

We define the Fibonacci numbers by the following recurrence:

F0 D 0 ;

F1 D 1 ; (3.22)

Fi D Fi 1C Fi 2 for i  2 :

Thus, each Fibonacci number is the sum of the two previous ones, yielding the sequence

0; 1; 1; 2; 3; 5; 8; 13; 21; 34; 55; : : : :

Fibonacci numbers are related to the golden ratio  and to its conjugate y, which are the two roots of the equation

x²D x C 1 (3.23)

and are given by the following formulas (see Exercise 3.2-6):

 D 1 Cp

5

2 (3.24)

D 1:61803 : : : ;

y D 1 p 5 2

D :61803 : : : : Specifically, we have Fi D ⁱ yⁱ

p5 ;

which we can prove by induction (Exercise 3.2-7). Sinceˇˇyˇˇ< 1, we have ˇˇyⁱˇˇ

p5 < 1 p5

< 1 2 ; which implies that

FiD

ⁱ p5 C1

2

; (3.25)

which is to say that the i th Fibonacci number Fiis equal to ⁱ=p

5 rounded to the nearest integer. Thus, Fibonacci numbers grow exponentially.

Exercises

3.2-1

Show that if f .n/ and g.n/ are monotonically increasing functions, then so are the functions f .n/ C g.n/ and f .g.n//, and if f .n/ and g.n/ are in addition nonnegative, then f .n/  g.n/ is monotonically increasing.

3.2-2

Prove equation (3.16).

3.2-3

Prove equation (3.19). Also prove that nŠ D !.2ⁿ/ and nŠ D o.nⁿ/.

3.2-4 ?

Is the functiondlg neŠ polynomially bounded? Is the function dlg lg neŠ polynomi-ally bounded?

3.2-5 ?

Which is asymptotically larger: lg.lg^n/ or lg^.lg n/?

3.2-6

Show that the golden ratio  and its conjugate y both satisfy the equation x²D x C 1.

3.2-7

Prove by induction that the i th Fibonacci number satisfies the equality FiD ⁱ yⁱ

p5 ;

where  is the golden ratio and y is its conjugate.

3.2-8

Show that k ln k D ‚.n/ implies k D ‚.n= ln n/.

Problems

3-1 Asymptotic behavior of polynomials Let

p.n/ D Xd

i D0

a_inⁱ ;

where ad > 0, be a degree-d polynomial in n, and let k be a constant. Use the definitions of the asymptotic notations to prove the following properties.

a. If k  d , then p.n/ D O.n^k/.

b. If k  d , then p.n/ D .n^k/.

c. If k D d , then p.n/ D ‚.n^k/.

d. If k > d , then p.n/ D o.n^k/.

e. If k < d , then p.n/ D !.n^k/.

3-2 Relative asymptotic growths

Indicate, for each pair of expressions .A; B/ in the table below, whether A is O, o,

, !, or ‚ of B. Assume that k  1,  > 0, and c > 1 are constants. Your answer should be in the form of the table with “yes” or “no” written in each box.

A B O o  ! ‚

a. lg^kn n^

b. n^k cⁿ

c. p

n n^{sin n} d. 2ⁿ 2ⁿ⁼² e. n^{lg c} c^{lg n} f. lg.nŠ/ lg.nⁿ/

3-3 Ordering by asymptotic growth rates

a. Rank the following functions by order of growth; that is, find an arrangement g₁; g₂; : : : ; g₃₀ of the functions satisfying g1 D .g₂/, g₂ D .g₃/, . . . , g29 D .g30/. Partition your list into equivalence classes such that functions f .n/ and g.n/ are in the same class if and only if f .n/ D ‚.g.n//.

lg.lg^n/ 2^lgn .p

2/^{lg n} n² nŠ .lg n/Š .³₂/ⁿ n³ lg²n lg.nŠ/ 2²ⁿ n^{1= lg n} ln ln n lg^n n  2ⁿ n^{lg lg n} ln n 1

2^{lg n} .lg n/^{lg n} eⁿ 4^{lg n} .n C 1/Š p lg n lg^.lg n/ 2^{p2 lg n} n 2ⁿ n lg n 2^2nC1

b. Give an example of a single nonnegative function f .n/ such that for all func-tions gi.n/ in part (a), f .n/ is neither O.gi.n// nor .gi.n//.

3-4 Asymptotic notation properties

Let f .n/ and g.n/ be asymptotically positive functions. Prove or disprove each of the following conjectures.

a. f .n/ D O.g.n// implies g.n/ D O.f .n//.

b. f .n/ C g.n/ D ‚.min.f .n/; g.n///.

c. f .n/ D O.g.n// implies lg.f .n// D O.lg.g.n///, where lg.g.n//  1 and f .n/  1 for all sufficiently large n.

d. f .n/ D O.g.n// implies 2^{f .n/}D O 2^g.n/

. e. f .n/ D O ..f .n//²/.

f. f .n/ D O.g.n// implies g.n/ D .f .n//.

g. f .n/ D ‚.f .n=2//.

h. f .n/ C o.f .n// D ‚.f .n//.

3-5 Variations onO and ˝

Some authors define  in a slightly different way than we do; let’s use ¹ (read

“omega infinity”) for this alternative definition. We say that f .n/ D .g.n// if¹ there exists a positive constant c such that f .n/  cg.n/  0 for infinitely many integers n.

a. Show that for any two functions f .n/ and g.n/ that are asymptotically nonneg-ative, either f .n/ D O.g.n// or f .n/ D.g.n// or both, whereas this is not¹ true if we use  in place of.¹

b. Describe the potential advantages and disadvantages of using¹ instead of  to characterize the running times of programs.

Some authors also define O in a slightly different manner; let’s use O⁰ for the alternative definition. We say that f .n/ D O⁰.g.n// if and only if jf .n/j D O.g.n//.

c. What happens to each direction of the “if and only if” in Theorem 3.1 if we substitute O⁰for O but still use ?

Some authors define eO (read “soft-oh”) to mean O with logarithmic factors ig-nored:

O.g.n// D ff .n/ W there exist positive constants c, k, and ne 0such that 0  f .n/  cg.n/ lg^k.n/ for all n  n0g :

d. Define e and e‚ in a similar manner. Prove the corresponding analog to Theo-rem 3.1.

3-6 Iterated functions

We can apply the iteration operator^used in the lg^function to any monotonically increasing function f .n/ over the reals. For a given constant c 2R, we define the iterated function f_c^by

f_c^.n/ D min˚

i  0 W f^{.i /}.n/  c

;

which need not be well defined in all cases. In other words, the quantity f_c^.n/ is the number of iterated applications of the function f required to reduce its argu-ment down to c or less.

For each of the following functions f .n/ and constants c, give as tight a bound as possible on f_c^.n/.

f .n/ c f_c^.n/

a. n  1 0

b. lg n 1

c. n=2 1

d. n=2 2

e. p

n 2

f. p

n 1

g. n¹⁼³ 2

h. n= lg n 2

Chapter notes

Knuth [209] traces the origin of the O-notation to a number-theory text by P. Bach-mann in 1892. The o-notation was invented by E. Landau in 1909 for his discussion of the distribution of prime numbers. The  and ‚ notations were advocated by Knuth [213] to correct the popular, but technically sloppy, practice in the literature of using O-notation for both upper and lower bounds. Many people continue to use the O-notation where the ‚-notation is more technically precise. Further dis-cussion of the history and development of asymptotic notations appears in works by Knuth [209, 213] and Brassard and Bratley [54].

Not all authors define the asymptotic notations in the same way, although the various definitions agree in most common situations. Some of the alternative def-initions encompass functions that are not asymptotically nonnegative, as long as their absolute values are appropriately bounded.

Equation (3.20) is due to Robbins [297]. Other properties of elementary math-ematical functions can be found in any good mathmath-ematical reference, such as Abramowitz and Stegun [1] or Zwillinger [362], or in a calculus book, such as Apostol [18] or Thomas et al. [334]. Knuth [209] and Graham, Knuth, and Patash-nik [152] contain a wealth of material on discrete mathematics as used in computer science.

In Section 2.3.1, we saw how merge sort serves as an example of the divide-and-conquer paradigm. Recall that in divide-and-divide-and-conquer, we solve a problem recur-sively, applying three steps at each level of the recursion:

Divide the problem into a number of subproblems that are smaller instances of the same problem.

Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in a straightforward manner.

Combine the solutions to the subproblems into the solution for the original prob-lem.

When the subproblems are large enough to solve recursively, we call that the recur-sive case. Once the subproblems become small enough that we no longer recurse, we say that the recursion “bottoms out” and that we have gotten down to the base case. Sometimes, in addition to subproblems that are smaller instances of the same problem, we have to solve subproblems that are not quite the same as the original problem. We consider solving such subproblems as part of the combine step.

In this chapter, we shall see more algorithms based on divide-and-conquer. The first one solves the maximum-subarray problem: it takes as input an array of num-bers, and it determines the contiguous subarray whose values have the greatest sum.

Then we shall see two divide-and-conquer algorithms for multiplying n n matri-ces. One runs in ‚.n³/ time, which is no better than the straightforward method of multiplying square matrices. But the other, Strassen’s algorithm, runs in O.n^2:81/ time, which beats the straightforward method asymptotically.

Recurrences

Recurrences go hand in hand with the divide-and-conquer paradigm, because they give us a natural way to characterize the running times of divide-and-conquer algo-rithms. A recurrence is an equation or inequality that describes a function in terms

of its value on smaller inputs. For example, in Section 2.3.2 we described the worst-case running time T .n/ of the MERGE-SORTprocedure by the recurrence T .n/ D

(‚.1/ if n D 1 ;

2T .n=2/ C ‚.n/ if n > 1 ; (4.1)

whose solution we claimed to be T .n/ D ‚.n lg n/.

Recurrences can take many forms. For example, a recursive algorithm might divide subproblems into unequal sizes, such as a 2=3-to-1=3 split. If the divide and combine steps take linear time, such an algorithm would give rise to the recurrence T .n/ D T .2n=3/ C T .n=3/ C ‚.n/.

Subproblems are not necessarily constrained to being a constant fraction of the original problem size. For example, a recursive version of linear search (see Exercise 2.1-3) would create just one subproblem containing only one el-ement fewer than the original problem. Each recursive call would take con-stant time plus the time for the recursive calls it makes, yielding the recurrence T .n/ D T .n  1/ C ‚.1/.

This chapter offers three methods for solving recurrences—that is, for obtaining asymptotic “‚” or “O” bounds on the solution:

 In the substitution method, we guess a bound and then use mathematical in-duction to prove our guess correct.

 The recursion-tree method converts the recurrence into a tree whose nodes represent the costs incurred at various levels of the recursion. We use techniques for bounding summations to solve the recurrence.

 The master method provides bounds for recurrences of the form

T .n/ D aT .n=b/ C f .n/ ; (4.2)

where a  1, b > 1, and f .n/ is a given function. Such recurrences arise frequently. A recurrence of the form in equation (4.2) characterizes a divide-and-conquer algorithm that creates a subproblems, each of which is 1=b the size of the original problem, and in which the divide and combine steps together take f .n/ time.

To use the master method, you will need to memorize three cases, but once you do that, you will easily be able to determine asymptotic bounds for many simple recurrences. We will use the master method to determine the running times of the divide-and-conquer algorithms for the maximum-subarray problem and for matrix multiplication, as well as for other algorithms based on divide-and-conquer elsewhere in this book.

Occasionally, we shall see recurrences that are not equalities but rather inequal-ities, such as T .n/  2T .n=2/ C ‚.n/. Because such a recurrence states only an upper bound on T .n/, we will couch its solution using O-notation rather than

‚-notation. Similarly, if the inequality were reversed to T .n/  2T .n=2/ C ‚.n/, then because the recurrence gives only a lower bound on T .n/, we would use

-notation in its solution.

Technicalities in recurrences

In practice, we neglect certain technical details when we state and solve recur-rences. For example, if we call MERGE-SORT on n elements when n is odd, we end up with subproblems of sizebn=2c and dn=2e. Neither size is actually n=2, because n=2 is not an integer when n is odd. Technically, the recurrence describing the worst-case running time of MERGE-SORTis really

T .n/ D

(‚.1/ if n D 1 ;

T .dn=2e/ C T .bn=2c/ C ‚.n/ if n > 1 : (4.3) Boundary conditions represent another class of details that we typically ignore.

Since the running time of an algorithm on a constant-sized input is a constant, the recurrences that arise from the running times of algorithms generally have T .n/ D ‚.1/ for sufficiently small n. Consequently, for convenience, we shall generally omit statements of the boundary conditions of recurrences and assume that T .n/ is constant for small n. For example, we normally state recurrence (4.1) as

T .n/ D 2T .n=2/ C ‚.n/ ; (4.4)

without explicitly giving values for small n. The reason is that although changing the value of T .1/ changes the exact solution to the recurrence, the solution typi-cally doesn’t change by more than a constant factor, and so the order of growth is unchanged.

When we state and solve recurrences, we often omit floors, ceilings, and bound-ary conditions. We forge ahead without these details and later determine whether or not they matter. They usually do not, but you should know when they do. Ex-perience helps, and so do some theorems stating that these details do not affect the asymptotic bounds of many recurrences characterizing divide-and-conquer algo-rithms (see Theorem 4.1). In this chapter, however, we shall address some of these details and illustrate the fine points of recurrence solution methods.

在文檔中 ALGORITHMS INTRODUCTION TO (頁 74-89)